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number  of  iterations  required  can  be  quite  large,  this  aspect  must  be 


a serious  consideration  for  users  with  a limited  computer  budget  which 


might  necessitate  premature  termination  of  the  computations 


Group  theoretic  procedures  comprise  the  second  category  of  integer 
programming  algorithms.  (See  Gomory  (1965),  and  Gorry  and  Shapiro  (1971)*) 


The  group  theoretic  approach  begins  by  transforming  the  original  problem, 


subject  to 


into  the  equivalent  form, 


maximize 


subject  to 


c_B  N > 0,  and  the  columns  of 


A have  been  partitioned  into  basic  and  nonbasic  sets  B,  N respectively, 


One  then  works  with  problem  (7)  in 


for  some  dual-feasible  basis  B 


place  of  problem  (6).  This  approach  has  been  called  group  theoretic 


because,  dropping  the  nonnegativity  restrictions  on  the  variables  x , 

D 

(7)  can  be  shown  to  be  equivalent  to  the  group  problem. 


Usually,  an  optimal  basis  B is  chosen,  though  this  is  not  necessary 


maximize 


subject  to 


where  D =*  |det  B 


' pecialized  and  efficient  solution  procedures  exist 


for  solving  proMems  such  as  (8)  above 


The  third  category  of  integer  programming  algorithms,  and  the  one 


which  ha  received  the  most  attention,  consists  of  the  enumerative  pro 


s (1965,  1967),  Dakin  (1965),  Geoffrion  (1969),  Glover 


cedures 


(1965)  > 81  1 Lanti  and  Doig  (,1960).)  These  methods  employ  a branch -and 


bound  type  s< arch  procedure  for  the  optimal  solution  to  the  integer 


programming  problem,  and  computational  experience  to  date  has  shown 


them  to  be  the  most  consistently  effective  of  the  three  types. 
Consider  again  the  integer  programming  problem  (6),  and  let 


F = (x  > 0|Ax  = b,  x integer) 


be  the  set  of  feasible  solutions  to  (6).  As  with  the  cutting  plane 


algorithms,  the  standard  first  step  in  an  enumerative  procedure  is  to 


solve  the  associated  linear  programming  relaxation  and  check  the  optimal 


solution  - call  it  x - for  integrality.  The  procedure  terminates  if 


x satisfies  the  integer  restrictions,  since  this  solution  must  then 


be  optimal 


Usually,  of  course,  several  integer-restricted  variables  will  occur 


at  non-integer  levels  in  the  optimal  LP  solution,  and  further  analysis 


*'  ' • W ™ 1 WT?-,  ; ’.  -'•'  v ' ^ 


is  required.  In  accordance  with  branch-and-bound  search  strategy,  the 
set  of  feasible  solutions  FQ  to  (6)  is  partitioned  into  (at  least) 
two  exhaustive  and  mutually  exclusive  subsets  F^,  Fg.  Most  commonly, 
the  partitioning  is  accomplished  by  specifying  new  bounds  on  one  of  the 
variables  which  violates  the  integer  restrictions  in  the  optimal  LP 
solution  x.  That  is,  some  such  variable  x^  is  selected,  and  the  sets 
Fi>  Fg  are  formed  by 

F1  = FQ  O fxlxk  < [xk]},  Fg  = F0D  {x|xk  > [xk]  + 1}  . 


Thus,  two  new  problems  are  created: 


maximize  {cx|x  eF^  , 


maximize  {cx|x  e Fg)  . 


It  is  clear  that  since  F1\J  Fg  = FQ,  the  optimal  integer  solution  to 
one  of  these  problems  muslj  be  the  overall  optimum  to  the  original  problem 
(6).  One  of  these  problems,  say  (9),  is  treated  immediately,  while  the 
other  problem,  say  (10),  is  put  on  a list  and  stored,  to  be  returned  to 
and  considered  later. 

Problem  (9)  is  treated  in  exactly  the  same  way  as  the  original 
problem.  The  integer  restrictions  are  relaxed,  and  (9)  solved  as  a 
linear  program.^  If  the  optimal  LP  solution  satisfies  the  integer 
restrictions,  it  must  clearly  be  optimal  for  (9).  However,  as  the 
optimum  for  (9)  may  not  be  the  overall  optimum  for  (6),  it  then 


— In  practice  this  is  most  efficiently  done  by  applying  the  dual  simplex 
method,  starting  with  the  LP-optimal  basis  for  (6),  or  by  parametric  pro- 
gramming on  the  new  bounds. 


I 

would  remain  to  return  to  the  list  of  partitioned  problems  - this  list 
so  far  contains  only  the  problem  (10)  - and  examine  each  in  turn  to  see 
if  there  exist  any  better  integer  solutions. 

If  the  optimal  LP  solution  to  (9)  does  not  satisfy  the  integer 

restrictions,  the  feasible  set  F^  is  itself  partitioned  in  a similar 

fashion  to  the  partitioning  of  F^.  As  before,  one  of  the  two  associated 

problems  is  added  to  the  list,  to  be  considered  later,  while  the  other 

is  examined  next.  This  process  - reoptimization,  partitioning,  etc.  - 

is  continued  until  some  partitioned  set  F^  is  obtained  for  which  the 

associated  LP  optimum  is  integer-feasible  as  well,  or  it  is  demonstrated 

that  the  optimal  solution  to  (6)  cannot  lie  in  the  set  F-.  In  branch- 

J 

and-bound  terminology,  the  partition  Fj  is  said  to  be  fathomed,  and 
at  this  point  a new  problem  is  extracted  from  the  list  and  the  search 
for  a better  integer  solution  to  (6)  is  continued  in  exactly  the  same 
way.  At  any  given  point  in  the  search,  the  best  integer  solution  found 
"so  far"  is  called  the  incumbent.  The  search  is  terminated  when  the  list 
of  partitioned  problems  has  been  exhausted,  and  the  final  incumbent 
solution  must  then  be  optimal.  (However,  if  no  incumbent  solution  has 
ever  been  obtained,  it  follows  that  (6)  must  have  no  feasible  solutions.) 

The  task  of  demonstrating  that  a given  partition  F.  cannot  contain 

J 

an  improved  integer  solution  to  (6)  is  accomplished  by  computing  bounds 
on  the  optimal  objective  value.  If  x is  the  current  incumbent  solution, 
the  corresponding  objective  value  cx  provides  a lower  bound  on  the  maximal 
value  in  (6).  Upper  bounds  on  the  maximum  objective  value  attainable  by 
any  feasible  integer  solution  in  a given  partition  F^  of  F^  are 
easily  derived.  The  optimal  objective  value  obtained  in  solving  the 
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LP  problem  associated  with  F.  provides  one  such  bound.  If  this  bound 

is  not  greater  than  cx,  the  partition  is  fathomed,  since  then 

F.  clearly  cannot  contain  a better  integer  solution  to  (6)  than  x. 

J 

Alternatively,  F.  can  also  be  fathomed  if  the  associated  linear  program 

u 

is  shown  to  be  infeasible,  since  this  in  turn  implies  F^  = 0.  In  either 

case,  the  analysis  of  the  partition  F.  is  complete. 

J 

The  enumerative  procedure  outlined  above  can  be  conveniently  dia- 
grammed using  a tree-search  structure  in  which  each  of  the  nodes  represents 

a partitioned  subset  F..  (See  Figure  1.)  The  partitioning  process 

J 

is  represented  by  branches  in  the  tree  which  connect  a given  set  (node) 
to  its  two  partitioned  subsets  (descendent  nodes).  The  generic  name 
"bra nch-and -bound"  derives  from  this  visualization  of  the  procedure,  in 
conjunction  with  the  bounding  methods  discussed  above.  While  the  dis- 
cussion of  the  branch-and-bound  method  has  been  within  the  context  of 
the  integer  programming  problem  (6),  it  should  be  kept  in  mind  that 
this  type  of  "divide  and  conquer"  search  strategy  is  in  fact  much  broader 
in  its  range  of  applicability.  (See  Mitten  (1970)  for  a formulation 
of  the  branch-and-bound  procedure  in  more  general  terms.)  The  branch- 
and-bound  concepts  introduced  here  will  be  drawn  upon  again  in  a different 
context  in  later  chapters. 

In  addition  to  the  references  above,  introductory  expositions  and 
further  details  on  any  or  all  of  the  integer  programming  methods  discussed 
may  be  found  in  Garfinkel  and  Nemhauser  (1972),  Greenberg  (1971) > or 
Hu  (1969).  Surveys  of  integer  programming  algorithms  have  been  presented 
by  Balinski  (1965),  Lawler  and  Wood  (1966),  and  Geoffrion  and  Marsten 
(1972).  This  last  survey  deserves  special  mention  in  that  there  is  also 
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Figure  1:  Branch-and-bound  tree  structure  for  the 

general  integer  programming  problem 

a general  framework  developed  which  serves  to  both  unify  and  contrast 
the  ideas  underlying  the  three  classes  of  integer  programming  algorithms. 
The  present  discussion  has  in  part  followed  the  presentation  in  the 
Geoffrion  and  Marsten  survey. 

Despite  the  wide  variety  of  algorithmic  approaches  which  have  been 
proposed,  the  art  of  integer  programming  today  still  cannot  claim  the 
same  degree  of  success  as  has  been  enjoyed  in  the  field  of  linear  pro- 
gramming. Computational  surveys  of  existing  computer  codes,  such  as 
those  by  Trauth  and  Woolsey  (1969)  and  Geoffrion  and  Marsten  (1972), 
indicate  that  even  small  or  moderately-sized  integer  programming  problems 
can  sometimes  require  disproportionately  large  amounts  of  computation 
time.  Moreover,  it  has  generally  been  accepted  that,  even  with  the 
best  of  the  available  computer  codes,  solution  times  for  integer 
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fc  I 

it 


programming  problems  increase  exponentially  with  the  number  of  problem 
variables.  While  it  has  not  yet  been  demonstrated  that  it  is  impossible 
to  construct  an  integer  programming  algorithm  whose  solution  time  can 
be  bounded  by  a polynomial  function  of  the  number  of  variables,  none- 
theless, the  inception  of  a general-purpose  integer  programming  algorithm 
with  the  same  overall  degree  of  efficiency  as  the  simplex  method  seems 
an  eternally  elusive  goal. 

By  contrast,  some  particularly  notable  successes  have  been  enjoyed 
by  special-purpose  algorithms  specifically  designed  to  treat  limited 
classes  of  specially-structured  integer  programming  problems.  See, 
for  example,  Geoffrion  and  Graves  (1973) , Healy  (1964),  Reardon  (1974), 
or  Tomlin  (1970).  The  classes  considered  are  generally  of  significant 
practical  importance,  and  often  the  work  has  been  undertaken  with  a 
particular  application  in  mind.  In  the  face  of  the  current  state  of  the 
art  of  general  integer  programming  algorithms,  and,  on  the  other  hand, 
the  results  obtained  in  applications  using  highly  specialized  algorithms, 
this  latter  approach  would  seem  a most  viable  alternative.  In  fact, 
these  considerations  have  served  as  the  philosophical  basis  for,  and  have 
greatly  motivated  the  present  work. 

1.3  The  Decomposition  Principle;  A Historical  Perspective 

Linear  programming  models  with  the  block  angular  structure  arise 


1 


frequently  in  many  large-scale  applications.  In  a multidivisional 
corporation,  for  example,  the  activities  in  each  of  the  subdivisions 
are  often  quite  independent  of  the  activities  of  other  subdivisions. 
If  complete  independence  holds,  of  course,  the  optimal  operation  of 
each  of  the  subdivisions  can  be  treated  as  a separate  problem  in  its 
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own  right.  Most  commonly,  however,  the  subdivisions  must  compete  for 
a few  resources  (e.g. , capital,  labor)  on  which  there  are  corporate- 
wide limitations.  This  situation  naturally  leads  to  mathematical  models 
with  the  block  angular  structure. 

As  practical  block  angular  problems  tend  to  be  rather  large  in 
size,  sometimes  involving  thousands  of  constraints  and  many  thousands 
of  variables  overall,  solution  algorithms  which  proceed  by  decomposing 
the  problem  into  its  (smaller)  subproblems  and  a single,  coordinating 
"Master  Problem"  can  be  of  great  value.  Since  the  computational  effort 
required  to  solve  linear  programming  problems  increases  approximately 
as  the  cube  of  the  number  of  rows  in  the  problem,  decomposition  algorithms 
often  provide  more  efficient  solution  procedures  for  large,  structured 
linear  programs  than  do  the  more  straightforward  linear  programming 
techniques.  Moreover,  even  with  today's  largest  computers,  the  extreme 
size  of  many  block  angular  problems  can  exceed  available  in-core  (high 
speed)  memory  capacity.  Since  accessing  disks,  drums,  or  tapes  is  a 
mu'h  slower  process  than  in-core  memory  retrieval,  computation  time 
can  quickly  become  excessive  for  such  problems.  By  breaking  the  problem 
into  the  smaller,  more  tractable  subproblems,  however,  decomposition 
algorithms  can  substantially  reduce  the  number  of  accesses  required  of 
external  storage  devices,  thereby  further  enhancing  computational  effi- 
ciency. 

Because  of  the  practical  importance  of  block  angular  mathematical 
programs,  and  the  inherent  advantages  of  decomposition  algorithms,  much 
attention  has  been  devoted  towards  exploiting  this  special  structure 
when  all  the  problem  variables  are  assumed  to  be  continuous.  Analysis 
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began  with  the  pioneering  work  of  Dantzig  and  Wolfe  (i960),  who  considered 
the  problem  defined  by  (l)-(4),  that  is,  the  linear  programming  problem 
associated  with  (P).  Decomposition  is  achieved  by  expressing  all  feasible 
subproblem  solutions  as  convex  combinations  of  the  extreme  points  of  the 
convex  set  of  such  solutions. 

Briefly,  the  central  ideas  of  the  Dantzig-Wolfe  algorithm  are  as 
follows.  Let  F^  denote  the  set  of  feasible  solutions  to  the 
subproblem,  namely, 

F^  = x^  > 0],  j = 1,2,  ...  , N . (11) 

We  assume  for  the  present  discussion  that  F^  is  bounded. Let 

7^^  (k  = 1,  2,  ...  ,E.)  denote  the  kth  extreme  point  of  , 

k 0 

where  E.  is  the  (finite)  number  of  such  extreme  points.  Then  any 
J 

x(j)  e F^  ^ can  be  expressed  as 


,(d)  _ 7?  (jL(d) 

ktl  k k 


for  some  set  of  weights  , k = 1,  2,  ...  , E . , such  that 

k J 


t "i111  * 1 


and  wl.  > 0,  for  k = 1,  2,  ...  , E.  . 
x — j 


Moreover,  since  any  point  xVd  which  can  be  expressed  in  this  form  must 
be  in  F^ , it  is  seen  that  (12),  (13)  provide  an  equivalent  repre- 
sentation for  the  set  F^. 


Thus,  letting 


17  ! ( i ) 

— There  are  minor  modif ications  to  the  procedure  when  FV,J  / is  unbounded. 
See  Dantzig  and  Wolfe  (i960). 
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(l)-(4)  can  be  re-written  as. 


(M)  maximize  £ pj^wj^  + £ p[2^wP^  +",+  £ P^M^  > (16) 

subject  to 


k=l 


k.l  k ' kti  *k  "* 


JW*  iW i«W<»  (u) 


E1 

A k 


k=l 


E2 

£ « 

k=l 


(2) 

k 


E„ 

S 

k=l 


= 1 


= 1 


= 1 


(18) 


>0  for  j = 1,2,  ...  , N;  k = 1,2,  ...  , E.  . (19) 
k - .1 


Problem  (m),  which  Dantzig  and  Wolfe  have  called  the  Master  Problem,  has 

N 

only  m + N rows,  rather  than  the  m + m.  rows  of  (P),  although 

0=1  J 

in  general  it  may  have  many  more  columns  than  (P).  This  difficulty  is 
circumvented,  however,  by  using  the  revised  simplex  method  and  generating 
the  columns  of  (M)  only  as  they  are  needed.  More  specifically,  in  pricing 
out  the  current  basic  feasible  solution  of  (m)  at  each  iteration,  the 
simplex  method  requires  that  the  column  of  (M)  with  lowest  relative  cost 


. , . ....  .. 
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r 


(if  negative)-^  be  introduced  into  the  basis  next.  This  column,  call 


it  a , is  determined  by 


* 

77a  = 


mxn 


{ min 


1 < j < N 1 < k < E. 


(20) 


where  tt  is  the  current  vector  of  dual  multipliers  associated  with  the 
rows  (17)  of  (m).  The  outer  minimization  in  (20)  is  trivial,  while  the 
inner  minimization  is  accomplished  for  each  j,  using  (15),  by 

,(d) 


min 

1 < k < E. 

- - j 


too 


min  (7TA  J )7p  = min  p^J  x^J  , 

1 < k < E.  K 

subject  to 

B(j)x(j)  < p(«5) 


(21) 


where  = ttA^  , 


From  the  theory  of  linear  programming,  it  is  well 

^ to  the  subproblem 


known  that  at  least  one  optimal  solution  x 
(21)  will  indeed  be  an  extreme  point  7^^  of  the  set  F^  for  some 

K 

k = 1,  2,  ...  , E . • 

Balas  (1966)  has  presented  an  alternative  decomposition  algorithm 
for  block  angular  linear  programming  problems.  In  Balas'  work,  the  linking 
constraints  are  at  first  ignored  and  the  subproblems  solved  separately. 

Any  infeasibilities  in  the  resulting  solution  are  then  used  to  develop 
a modified  set  of  price  vectors  for  the  subproblems.  The  subproblems 
are  resolved  with  the  new  objective  functions,  and  the  procedure  con- 
tinues to  iterate  in  this  manner  until,  after  a finite  number  of  such 
iterations,  the  optimal  solution  is  obtained. 


— ^ If  all  columns  are  found  to  have  nonnegative  relative  costs,  the 
current  solution  is  optimal. 


1 ■ 
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Balas'  decomposition  algorithm  bears  many  similarities  conceptually 
to  the  algorithm  of  Dantzig  and  Wolfe.  Both,  for  example,  provide  a 
means  for  decentralized  planning  in  a multi-divisional  organization 
through  the  use  of  an  internal  pricing  system.  However,  while  in  Dantzig 
and  Wolfe* s procedure,  a feasible  solution  is  obtained  first  and  then 
iteratively  improved  upon  until  the  optimum  is  achieved,  Balas*  algorithm 
proceeds  by  obtaining  an  ''optimal"  (infeasible)  solution  first,  then 
iteratively  reducing  the  infeasibility  of  the  generated  solutions  while 
maintaining  optimality. 

Others  who  have  considered  the  problem  of  decomposing  structured 
linear  programs  are  Bennett  (19 66),  Hartman  and  Lasdon  (1970),  Ritter 
(1967),  and  Rosen  (1964).  To  date,  however,  little  consideration  has 
been  given  to  block  angular  models  in  which  the  decision  variables  are 
required  to  take  on  only  integer  values.  Yet,  especially  in  integer 
programming  problems,  where  computation  time  may  increase  exponentially 
with  the  number  of  problem  variables,  it  appears  that  substantial  savings 
can  be  achieved  by  solving  a judiciously  chosen  sequence  of  the  smaller 
subproblems,  rather  than  attacking  the  problem  as  a whole.  For  example, 
it  should  be  possible  to  solve  ten  ten-variable  subproblems  in  much  less 
computer  time  than  would  be  required  to  solve  one  one-hundred  variable 
problem.  Moreover,  as  with  its  linear  programming  counterpart,  an 
algorithm  for  solving  large,  block  angular  integer  programs  which  treated 
only  one  subproblem  at  a time  could  render  solvable  problems  whose  overall 
size  would  otherwise  result  in  prohibitive  in-core  storage  requirements. 
These  considerations  have  motivated  the  present  work. 

At  this  point,  previous  and  related  results  in  the  area  of  decomposing 
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integer  programs  deserve  some  mention.  Schrage  (1973)  was  apparently 
the  first  to  consider  the  advantages  of  a decomposition  algorithm  for 
specially  structured  integer  programming  problems.  Schrage* s paper  was 
primarily  concerned  with  problems  in  which  the  constraint  matrix  possesses 
a dual  angular  structure,  that  is,  where  the  subproblems  are  tied  to- 
gether by  a set  of  linking  columns  rather  than  linking  constraints. 

In  a linear  programming  sense,  this  is  the  dual  structure  to  the  one 
considered  here. 

Reardon  (197*0  showed  how  certain  capital  budgeting  problems  could 
be  formulated  with  the  dual  angular  structure,  and,  developing  and 
refining  Schrage* s ideas,  designed  a decomposition  algorithm  to  treat 
problems  with  this  structure.  Schrage *s  suggestion  was  that,  in  using 
a partial  enumeration  solution  procedure  for  dual  angular  integer  pro- 
grams, one  should  give  priority  to  fixing  the  values  of  the  linking 
variables  first.  Once  these  columns  have  been  fixed,  the  problem  de- 
composes of  its  own  accord  into  independent  subproblems.  Reardon's 
idea,  then,  was  to  separate  the  problem  into  subproblems  from  the  start, 
thereby  relaxing  the  restriction  that  the  linking  variables  (called 
"first  stage"  variables)  must  take  on  the  same  values  in  each  subproblem. 
This  restriction  is  implicitly  re-imposed  by  stipulating  that  each  of 
the  subproblems  must  have  identical  first-stage  search  trees.  In  compu- 
tational tests  of  the  algorithm,  Reardon  obtained  an  approximately  linear 
increase  in  solution  time  with  problem  size,  a vast  improvement  over 
the  roughly  exponential  growth  observed  with  a non-decomposition  approach! 

It  is  natural  to  consider  using  Lagrangean  relaxation  as  a means 
of  decomposing  block  angular  integer  programs.  Such  an  approach  would 


' 
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be  analogous  to  Balas'  infeasibility  pricing  algorithm  for  decomposing 
block  angular  linear  programming  problems.  Geoffrion  (197*0  discusses 
the  uses  of  Lagrangean  relaxation  in  integer  programming.  The  central 


idea  is  to  simplify  an  otherwise  difficult  problem  by  using  a suitably 
chosen  set  of  multipliers  to  take  any  "complicating"  constraints  (in 
the  present  context,  these  would  be  the  linking  constraints)  up  into 
the  objective  function.  Thus,  using  the  m-vector  of  multipliers 
\ > 0,  (P)  would  be  transformed  to, 


maximize 


subject  to 


£ c(j)x(j)  + \(b  - 2 A(J)x('3))  , 
j=l  0=1 


B<2)x<2> 


bWx<N)  < p(H) 


By  choosing  specific  values  \ for  the  multipliers,  (22) -(25)  then 
immediately  decomposes  into  the  N subproblems : 

maximize  (c^  - \*A^)x^  , 


subject  to 


B(o)x(o)  < ^(o) 


> 0, 


integer  , 


Lasdon  (1973)  has  considered  an  approach  along  these  lines  in  a 
production  scheduling  application.-^  The  difficulty  with  such  an  approach, 
as  Lasdon  points  out,  is  that  in  general,  due  to  the  non-convexity  of 
the  feasible  region,  there  is  no  guarantee  that  there  exists  a choice 
of  values  for  the  multipliers  X that  leads  to  an  optimal  solution 
to  (P).  Worse,  one  cannot  in  general  even  be  assured  of  finding  a feasible 
solution  - that  is,  one  satisfying  the  ‘’inking  constraints  as  well  as 
the  subproblem  constraints  - for  (P)  in  this  way.  In  view  of  Bala s’ 
results  with  block  angular  linear  programs,  this  state  of  affairs  is 
somewhat  disappointing.  However,  Lagrangean  relaxation  may  still  be 
useful  as  a heuristic  approach  in  integer  programs  where  it  is  not 
strictly  required  that  all  of  the  constraints  be  met. 


1.4  Overview 

In  the  remaining  chapters,  then,  an  exact  algorithm  for  the  decom- 
position of  block  angular  integer  programs  is  developed,  A conceptual 
outline  of  the  algorithm  is  presented  in  Chapter  2.  Decomposition  occurs 
through  resource  allocation,  and  a branch-and-bound  search  for  the  optimal 
allocation  of  linking  resources  to  subproblems  is  established. 

Details  of  the  algorithm  are  discussed  in  Chapters  3-5.  Specifically, 
Chapter  3 deals  with  the  question  of  how  the  branching  is  to  be  done, 
while  Chapters  4 and  5 develop  procedures  for  computing  bounds  at  each 
node  in  the  search  tree.  In  Chapter  4,  the  case  in  which  (P)  has  only 
a single  linking  constraint  is  considered.  In  modeling  the  operation 
of  a multi-divisional  organization,  such  a constraint  might  represent 


Lasdon's  paper,  as  well  as  Geoffrion's,  also  contains  an  excellent 
discussion  of  the  concepts  relating  to  generalized  Lagrange  multipliers. 
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the  overall  corporate  budgeting  limitation  to  which  all  of  the  subdivisions 
(subproblems)  are  subject.  The  simplifications  that  result  in  this  case 
give  rise  to  a specially  structured  master  problem,  and  an  efficient 
solution  algorithm  which  exploits  this  structure  is  developed.  The 
problem  of  computing  bounds  in  the  presence  of  more  than  one  linking 
constraint  is  addressed  in  Chapter  5» 

Computational  results  obtained  with  a Fortran  code  of  the  decompo- 
sition algorithm  on  an  IBM  370/l68  are  presented  in  Chapter  6.  Finally, 
in  Chapter  7,  we  summarize  our  results  and  discuss  some  suggestions  for 
areas  of  future  research. 

Some  of  the  results  of  Chapter  4 are  generalized  in  Kochman  (1976), 
where  a class  of  nonlinear  integer  programming  problems,  of  which  the 
master  problem  developed  in  Chapter  4 is  a member,  is  introduced.  A 
general  solution  procedure  for  problems  in  this  class  is  discussed,  and 
two  illustrative  examples  are  presented. 


p 


$ 
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CHAPTER  2 

RESOURCE  DECOMPOSITION:  A CONCEPTUAL  OUTLINE 

The  algorithms  of  Dantzig  and  Wolfe  (i960)  and  Balas  (1966)  dis- 
cussed in  Chapter  1 both  utilize  pricing  mechanisms  to  achieve  the 
decomposition  of  block  angular  linear  programs.  Lasdon's  generalized 
Lagrange  multiplier  approach  to  the  integer  programming  problem  might 
also  fall  into  the  category  of  "price  decomposition"  algorithms.  The 
difficulty  with  such  approaches  in  an  integer  programming  context,  however 
is  that,  in  sharp  contrast  to  the  linear  programming  case,  there  does 
not  exist  any  comparably  well-defined  and  well-behaved  system  of  shadow 
prices.  Consequently,  an  alternative  approach  is  considered  here,  and 
the  resulting  algorithm  proceeds  through  what  might  be  called  resource 
decomposition.  In  this  chapter,  we  present  a conceptual  basis  for  the 
decomposition  algorithm. 
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Let  x^,  1 < j < N,  denote  an  optimal  solution  to  (P) , and,  corres- 

pondingly,  define  the  m-vectors  b'u  by 

b(J)  = a^x^,  j = 1,  2,  ...  , N . (6) 

Conceptually,  the  linking  constraints  (2)  can  be  viewed  as  repre- 
senting common  resources  which  the  subproblems  must  share.  Under  this 
interpretation,  the  b'J/  in  (6)  are  seen  to  be  in  some  sense  an  optimal 
allocation  of  these  resources  to  the  subproblems.  Clearly,  were  the 
values  , j = 1,  2,  ...  , N,  known  beforehand,  (P)  could  be  decom- 

posed  immediately  and  the  optimal  solution  xw  , j = 1,  2,  ...  , K, 
obtained  simply  by  solving  the  N subproblems, 

(S.)  maximize  f 

J 

subject  to 

a(j)x(j)<;cj) 

B(-3)x(.i)  < p(J) 

> 0,  integer  , 

for  j = 1,  2,  ...  , N.  These  subproblems  (S^),  j = 1,  2,  ...  , N, 
could  be  solved  by  any  of  the  various  standard  integer  programming  algor- 
ithms discussed  in  Chapter  1. 

Unfortunately,  in  general  an  optimal  allocation  bv  , 
j = 1,  2,  ...  , N,  is  not  known  until  after  (p)  has  been  solved.  How- 
ever, in  view  of  the  reduction  which  results,  it  appears  that  it  may  be 
profitable  to  "guess"  at  values  for  the  b'  , j = 1,  2,  ...  , N,  or 
rather,  to  devise  a systematic  means  of  determining  a sequence  of  such 
guesses  which  leads  to  an  optimal  allocation  in  a finite  number  of  steps. 


20 


i 


To  this  end,  instead  of  considering  the  b 


(d) 


j — 1,  2,  • < . , N,  as 


fixed  vectors,  we  introduce  into  the  problem  the  additional  allocation 


each  b 


id 


,<J>  - 

has  the  interpretation 


variables  b'*"  = (b^,  b£y  ...  , b .),  j = 1,  2,  ...  , N,  where 


b.  . = amount  of  i 
ID 


.t.h 


linking  resource  allocated  to 


,th 


subproblem,  1 < i < m,  1 < j < N . 


The  resulting  (equivalent)  formulation  of  the  original  problem  (P) 
is 


(P‘) 


. • (1)  (1) 
maximize  cv  xv 

f 

(7) 

subject  to 

b^ 

+ b^  +•••+  b^ 

< b 

(8) 

A(1)x(1)_b(1) 

< 0 ^ 

<B(1) 

A<2>*(2)-b<2) 

< 0 

b<2>*<2> 

CM 

oa 

VI 

-(9) 

A(B)x(H).b(B) 

< 0 

B<B)x<K) 

-\ 

CO. 

VI 

x^  > 0,  integer,  for  j * 1,  2,  ...  , N . 

(10) 

The  transformed  problem  (P*)  is  itself  a block  angular  integer  program, 
in  which  the  linking  constraints  are  now  of  the  form  (8).  These  con- 
straints on  the  b^\  j = 1,  2,  ...  , N,  are  necessary  to  insure  that 


k 


d 
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the  linking  resources  are  not  over-allocated,  and  are  implied  by  the 
inequalities  (2)  in  (P). 


' 

I 

£ 

I Although  they  may  be  required  to  be  nonnegative  in  certain  appli- 

cations (e.g.,  when  they  actually  represent  resource  allocations),  the 
allocation  variables  b^,  j = 1,  2,  ...  , N,  are  not,  in  general, 
restricted  in  sign.  We  shall  assume  throughout,  however,  that  each  of 
the  matrices  A^,  j = 1,  2,  ...  , N,  has  all  integer  entries.  This 
assumption  implies  that  any  feasible  integer  solution  to  the  jth 

subproblem  of  (P)  can  use  only  integeral  amounts  of  the  linking  resources. 
Consequently,  the  search  for  an  optimal  allocation  of  these  resources 
to  the  subproblems  can  be  restricted  to  integer  values  of  the  allocation 
variables  b^.— 

Reordering  the  variables  in  (P'),  and  writing  the  problem  in  block 
form  leads  to  the  representation. 


x^J  > 0,  integer,  for  j = 1,  2,  ...  , N 
-in  + orToy.  fQr  j = 1,  2,  ...  , N 


integer. 


1 


In  this  form,  (P1)  takes  on  the  dual  angular  structure  considered  by- 
Reardon  (1974),  with  the  allocation  variables  forming  the  linking  columns. 
In  this  sense,  the  algorithm  developed  here  is  a specialized  procedure 
for  decomposing  dual  angular  integer  programs  in  which  the  linking 
columns  possess  the  special  structure  exhibited  in  (P").  The  out- 
standing feature  of  this  structure  is  that  each  of  the  linking  variables 
b. . has  a non-zero  coefficient  in  exactly  one  subproblem.  Whereas  in 
the  general  dual  angular  problem  all  linking  columns  must  be  assigned 
fixed  values  before  decomposition  occurs  (at  which  point  the  decompo- 
sition is  total),  in  (P")  partial  decomposition  occurs  whenever  all  the 

allocation  variables  b^  associated  with  one  given  subproblem  are 

* 

/ . \ / . \ ^ 

fixed.  Once  values  V = b'  (possibly  nonoptimal)  have  been 

specified,  the  subproblem  (S.)  can  >-.e  solved  immediately,  independently 

3 

of  the  others,  and  the  information  obtained  used  to  improve  the  allo- 
cations in  the  remaining  subproblems.  This  point  is  clarified  in 
Chapter  5. 


2.2  A Branch -and -Bound  Search  for  an  Optimal  Allocation 


Using  the  concepts  of  resource  decomposition  described  in  the  pre- 
ceding section,  the  block  angular  integer  program  (P)  is  transformed  into 

the  N "separate"  subproblems  (S.)  by  specifying  values  b'J'  for 

3 


j = 1,  2,  ...  , N,  is  sufficient  for  this  conclusion,  but  not  necessary. 
It  may  happen  in  certain  applications,  for  example,  that  the  nature  of 
the  linking  resources  is  such  that  they  can  be  meaningfully  divided  among 
the  subproblems  only  in  integral  amounts. 
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the  allocation  variables  b 


finding  an  optimal  solution  x 


P)  thus,  becomes 


that  of  determining  an  optimal  allocation  of  linking  resources  to  sub 


problems.  The  subproblems  are  tied  together,  and  the  possible  choices 


of  values  b 


inequalities  (8).  Moreover,  under  the  assumption  of  integrality  in 


, N,  only  integer  values  for  the 


allocation  variables  need  be  considered 


In  many  practical  situations,  previous  experience  with  the  problem 


may  provide  a good  initial  estimate  for  an  optimal  allocation.  The 


difficulty,  of  course,  is  that  although  the  subproblem  optimal  solutions 


corresponding  to  a given  allocation  b 


may  constitute  a very  good  - perhaps  even  optimal  - solution  to  (P), 


there  is  no  guarantee  that  these  solutions  will  be  optimal,  nor  even 


any  simple  means  of  determining  their  optimality  or  nonoptimality 


Consequently,  it  will  generally  be  necessary  to  examine  a sequence  of 


estimates  for  the  optimal  allocation.  Obviously,  it  is  desirable  to 


design  this  sequence  in  such  a manner  that  only  a "reasonable"  number 


of  allocations  must  be  examined  in  all 


The  procedure  developed  here  utilizes  a branch  -and-bound  type  search 


strategy  to  locate  an  optimal  allocation,  and  is  similar  in  concept  to 


that  presented  in  Section  1.2  in  the  context  of  the  general  integer 


programming  problem.  In  the  present  context,  the  feasible  set  F 


to  be  searched  is  defined  by 


f i ) T — 

where  b = ’ braj  ^ ’ j = 1>  2>  • • • > N,  b^,  b^  represent 

lower  and  upper  bounds,  respectively,  on  the  allocation  variable  b. ., 

1 < i < m,  1 < j < N,  and  b is  the  vector  of  linking  constraint 

right-hand  sides  in  (2).  Procedures  for  computing  the  initial  values 

of  the  b. . and  b. . are  presented  in  Section  2. J. 

i J ij  3 , 

The  basic  structure  of  the  search  tree  in  this  case  is  shown  in 

f -j  ) * £ # m 

Figure  1.  To  begin  the  search,  an  allocation  b'0'  '=  (b, ...  , b ,)  , 

v lj ’ ’ mj'  ’ 

j = 1,  2,  ...  , N,  is  chosen  from  the  set  FQ  and  substituted  into  the 

subproblems  (S.).  Each  of  the  subproblems  is  solved  in  turn  to  obtain 
3 r * * 

its  integer  optimum  , j = 1,  2,  . . . , N ("optimal"  relative 

to  the  allocation  b'J  ).  If  all  the  subproblems  are  feasible  under 
the  allocation  b ^ ' , j = 1,  2,  ...  , N,  the  corresponding  objective 
* N (j)  (j)* 

value  z = 2./  c x'  immediately  provides  a lower  bound  for  the 

j=l 

optimal  objective  value. 


Node  0 


bi  j 

oJo  0^0 


Node  1 


\ bj  j > b!  i 

\^xo^o  xo^o 
\ 


Node  2 


Figure  1:  Search  tree  structure  for  locating  an  optimal  allocation 
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At  this  point  it  is  unknown  whether  x 


is  optimal  for  (P)  in  an  overall  sense.  We  therefore  seek  an  improved 


solution,  or,  by  failing  to  find  one,  verification  of  the  optimality  of 


N.  Assuming,  then,  that  x 


is  not  optimal  for  (P),  at  least  one  of  the  allocations  b 


must  be 


nonoptimal.  Hence,  one  variable,  b^  . say,  is  chosen  as  the  branching 

V o 

variable,  and  F is  partitioned  as  indicated  in  Figure  1.  On  one  branch, 


a new  lower  bound  on  b,  . , say  b,  (where  b'  .. 

O^O  0^0  O' 

specified.  On  the  other  branch,  the  upper  bound  on  b 


(where  b 


are  defined 


In  keeping  with  the  "divide  and  conquer"  strategy  of  branch-and-bound 


ijiust  be  chosen  to  ensure  that  an 


search,  the  new  bounds  b ’ . , bl^ 

1oJo  1oJo 

optimal  allocation  lies  in  the  set  ] 


while 


before,  one  of  the  two  sets 


is  examined  next 


investigation.  The  other  set,  F 


The  examination  of  a given  subset  F . begins  with  the  computation 

of  an  upper  bound  z on  the  optimal  objective  value  in  (P)  attainable 

. (i)  (m).  — * . 

by  any  allocation  (bv  , ...  , bv  ) e F..  If  z < z (the  current 

3 

incumbent  objective  value),  F.  is  fathomed  and  a new  problem  is  taken 


The  branching  process, 
is  discussed  in  Chapter  3 


In  particular,  we  must  have  b’  . > b:  . . 

^ p iB«i  o 

including  the  selection  of  these  revised  bounds 


from  the  list  instead.  If  z > z , however,  F.  may  contain  an  allo- 

u 

cation  leading  to  an  improved  objective  value  in  (P).  In  this  case, 


a promising  new  allocation  (b 


(l)'  ,(N)\  , . 

, ...  , bN  ) is  chosen  from  F 


and  substituted  into  the  subproblems.  From  this  point,  the  procedure 

that 

(i)’ 


for  Fj  is  the  same  as  that  described  above  for  F^,  with  the  exception 


that  the  new  solution  x 

N 


, j = 1,  2,  . . . , N,  and  corresponding 


objective  value  z‘  = £ c(j)x(j)  is  retained  as  the  (new)  incumbent 


j=l 

. * 

solution  only  if  z'  > z . 

We  summarize  these  remarks  with  a conceptual  outline  of  the  algor- 
ithmic procedure : 


Rudimentary  Branch-and-Bound  Algorithm  (RBBA) 


Step  1. 


Initialize  the  list  to  contain  only  the  original  set  F^. 
* 

Set  z = 


Step  2. 


If  the  list  is  empty,  stop: 

If  z = -«>,  (p)  is  infeasible. 

Otherwise,  the  incumbent  is  optimal  and  z 
objective  value. 

If  the  list  is  not  empty,  select  one  of  the  sets 
searched  next,  and  remove  Fj  from  the  list. 


is  the  maximal 


Fj  to  be 


Step  3- 


(The  bounding  step.)  Compute  an  upper  bound  U 
objective  vt_lue  attainable  by  any  allocation  in 
If  U < z , go  to  Step  2.  (F.  is  fathomed.) 

Otherwise,  select  a promising  allocation  (b'  , 

and  go  to  Step  4. 


on  the  maximal 


€ F. 
0 
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Step  4.  Using  the  allocation  b^  , solve  the  jth  subproblem  (S.) 

d 


to  obtain  the  integer  optimum  x 
J = 1,  2,  ...  , N. 


(j) 


(if  it  exists), 


(i)  If  any  subproblem  is  infeasible,  or  if  the  current  allo- 


(N) 


cation  (bv'L'  , ...  , bv‘”/  ) can  otherwise  be  recognized 
as  nonoptimal,  go  (immediately)  to  Step  5-  (Don't  solve 
the  remaining  subproblems.) 

(ii)  If  all  subproblems  (s^)  are  feasible,  and.  (S^)  is 
unbounded  for  some  1 < k < N (i.e.,  (S^)  has  no 

finite  optimal  solutions),  stop.  (Problem  (p)  is  un- 
bounded  and  the  current  allocation  (b'  ,...,b'  ) 

is  clearly  optimal. ) 

(iii)  If  all  subproblems  (s.)  have  finite  optimal  solutions 


x(j),  , j = 1,  2, 


N 


objective  value  £)  c(j)x(d) 


, N,  and  if  the  corresponding 
* 


> z 


reset 


N 


j=l 


.OUJ)* 


,(J) 


= £ cvo/xxo/  and  install  x 

N 


j=l 


, j - 1,2, . . . ,N, 


as  the  new  incumbent,  (if  c 

current  incumbent  is  retained. ) 


(dL(d)  * 


< z , the 


Go  to  Step  5« 


Step  5.  (The  branching  step.)  Choose  a branching  variable  b..  and 


o”o 


partition  the  set  F.  as  discussed  above:  e.g., 

J 

(i)  In  the  first  subset,  F.  , reset  b.  . = b*  . 

-ij_  -ij 


_o  o 

(ii)  In  the  second  subset,  F,  , reset  b.  , = bf  . 

Jo  x0d  o 1-d 


o o 

oV 


Add  F 


j i 


to  the  list. 


Then  go  to  Step  3>  where  now  F.  takes  the  place  of  F 


j. 


'J* 
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The  description  of  the  branch-and-bound  algorithm  above  is  far  from 
complete.  Procedures  for  computing  the  objective  function  bounds  required 
at  Step  3 are  the  subjects  of  Chapters  4 and  5-  It  will  be  seen  there 
that  the  problem  of  locating  a "promising"  new  allocation  in  an  unfathomed 


set  F.  in  Step  3 is  closely  connected  with  the  bounding  procedures. 
I 


In  fact,  the  allocation  selected  for  use  in  Step  4 will  be  the  one  in 


F.  which  gives  the  greatest  upper  bound  on  the  overall  objective  function 
J 


value  in  (P) . 

In  Step  5,  the  two  points  requiring  further  specification  are  deciding 
which  allocation  variable  should  be  selected  as  the  branching  variable, 
and,  once  chosen,  specifying  the  bounds  b!  . , b’  . which  define  the 


o o o o 

partitions  F.  , F.  . These  are  discussed  in  the  next  chapter. 


'V  J2 


Finally,  some  decision  rule  is  needed  in  Step  2 regarding  the  selection 


of  the  subset  F.  from  the  list.  In  the  algorithm  developed  here,  the 
J 


last-in-first-out  (LIFO)  rule  is  used.  Under  the  LIFO  strategy,  the 


last  set  F.  added  to  the  list  in  Step  5 is  always  the  first  one  to 
J 


be  chosen  for  examination  next  in  Step  2.  This  rule  facilitates  pro- 
gramming the  algorithm,  and,  more  importantly,  can  increase  algorithmic 
efficiency  by  reducing  the  computational  effort  required  in  reoptimization. 
For  these  reasons,  use  of  the  LIFO  strategy  is  a standard  practice  in 
many  branch-and-bound  type  search  procedures. 

The  advantages  of  the  decomposition  procedure  become  apparent  in 
Step  4.  For  the  reasons  discussed  in  Chapter  1,  the  most  significant 


feature  of  the  algorithm  is  the  fact  that  only  the  subproblems  (S^) 


ever  need  be  solved  explicitly.  Moreover,  it  will  be  seen  that  the 
subproblems  may  be  solved  by  any  linear-programming  based  method.  This 
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is  advantageous  if  the  subproblems  themselves  possess  structures  that 


can  be  efficiently  exploited  by  specialized  solution  procedures.  Finally 

. * 

it  will  usually  be  possible  to  recognize  that  an  allocation  b J , 


, N,  is  nonoptimal  without  solving  all  of  the  subproblems 


When  this  situation  occurs,  the  computations  in  Step  are  terminated 


and  we  proceed  immediately  to  Step  5 


Computing  Bounds  on  the  Allocation  Variables 


For  most  problems,  the  initial  specification  of  the  bounds  b 


(1  < i < ®,  l<j<N)  defining  the  set  Fn  is  not  crucial,  provided 


of  course  that  the  intervals 


, b ] are  chosen  sufficiently  wide 


to  ensure  that  an  optimal  allocation  is  included.  In  this  section  we 


present  some  simple  procedures  for  computing  valid  initial  bounds  on 


the  allocation  variables 


Suppose  first  that  we  have  been  given  upper  and  lower  bounds  U 


sub problem 


The  bounds  L'  , IT  in  (19)  often  can  be  readily  inferred  from  the 
functional  constraints,  and,  inasmuch  as  many  practical  integer  programming 
algorithms  require  the  specification  of  such  bounds  prior  to  solution 


of  the  problem,  the  assumpti  Is  not  a very  restrictive  one 


Define  the  index  sets  P 


N,  where  A 


the  entry  in  the 


column  of  the  matrix  A 


of  linking 


row 


iubproblem.  Then  for  1 < i < m 


represents  a valid  lower  bound  on  the  allocation  variable  b 


Similarly,  it  is  easy  to  see  that 


defines  a valid  upper  bound  for  b 


sets  P, N. . is  empty,  the  corresponding  sum  is  taken  to  be  zero 


Both  the  bounds  (21),  (22)  are  valid  in  the  sense  that  the  bounds 


imply  that  the 


subproblem  must  "use"  at  least  b.  . units  of  the  i 


linking 


resource,  and  cannot  possibly  use  more  than  b 


units.  Thus,  there 


must  exist  an  optimal  allocation  of  linking  resources  to  subproblems 


Having  already  computed  the  lower  bounds  b. . from  (21),  the  upper 
bounds  b, , in  (22)  can  possibly  be  tightened,  using  (8),  since 


jgWpNnt  If^WK^MBa  ■ « n^i  »»tji 


5)  b..  < b.  ==>  b,  < b.  - b.. 

A ik  - 1 1J  - 1 k/3  lk 


-b>-£> 


Thus, 


b'j  = rninfb^,  b.  - £ blk),  l<i<m,  l<j<N,  (2 

kpj 

gives  at  least  as  tight  an  upper  bound  on  b„  as  (22)  alone. 

There  are  three  special  cases  of  (21),  (22)  which  deserve  further 
comment  due  to  the  frequency  with  which  they  arise.  Firstly,  if,  as 


will  usually  be  the  case,  we  have  L 


(nonnegativity),  (21),  (22)  reduce  to 


= 0 


for  j = 1,  2,  ...  , N 


= 2 a5J}u[J),  and  b^  = |)  A^}u£j),  l<i<m,  l<j<N 


Moreover,  in  many  practical  problems  the  decision  variables  x'J/  are 
restricted  to  take  on  only  the  values  0 or  1.  In  this  case,  (24)  simpli- 
fies even  further  to 


feu  = 2 Aik 

j£Nij 


2 A(J},  and  b = 2 A$J},  l<i<m,  l<j<N.  (25) 


iJ  -U  ik 


Finally,  in  many  applications  (e.g.,  in  capital  budgeting  problems)  all 

the  constraint  coefficients  are  nonnegative.  From  (24),  it  is  clear 

that  if  = 0 for  some  j and  1 < i < m (i.e.,  > 0),  and 

there  are  nonnegativity  restrictions  on  the  (L^  = 0),  then 

b . . = 0 for  1 < i < m. 

-ij  - - 
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So  far  we  have  considered  only  the  bounds  on  the  decision  variables, 
and,  to  some  extent,  the  implications  of  the  linking  constraints  (8) 
in  computing  upper  and  lower  bounds  on  the  allocation  variables.  Tighter 
bounds  than  those  given  by  (21)  or  (22)  may  be  obtained  (at  greater 
computational  expense,  of  course)  by  using  the  subproblem  constraints 
(3)  as  we  Hi  For  1 < i < m and  1 < j < N,  consider  the  problem, 

n . 


maximize 
subject  to 


j 

D 

k=l 

A(d)„(j) 

Aik  xk  ' 

(26) 

B(j)x(j)  < p(^ 

(27) 

L(j)  < x(j/  < y( j) 

(28) 

x(j)  integer  . 

(29) 

By  the  same  reasoning  as  before,  the  optimal  solution  x 
(29)  provides  a valid  upper  bound  on  b^.  by 


(J)’ 


to  (26)- 


n . 
J 


u £ ik  k 


(30) 


Similarly,  if  the  objective  function  (26)  is  minimized  in  (26)-(29),  the 


resulting  optimal  solution  x 


(j> 


provides  a lower  bound  on  b. . by 

• J 


(3D 


While  the  bounds  b.  b given  in  (30),  (31)  will  generally  be 
i J 

much  tighter  than  those  provided  by  (21),  (22),  the  extra  computational 
effort  required  can  be  considerable,  since  it  involves  solving  2mN 
integer  programs.  The  work  required  can  be  mitigated  to  some  extent 
if  an  enuuerative  algorithm  is  used.  In  particular,  note  that  for  fixed 
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j the  feasible  region  defined  by  (27) -(29)  is  the  same  for  all  i,  so 
feasible  solutions  found  in  the  course  of  solving  (26)-(29)  for  one  i 
can  be  used  as  initial  incumbents  in  solving  (26) -(29)  for  different  i 


Nonetheless,  for  most  problems  it  is  doubtful  that  the  work  expended 
to  generate  the  bounds  (30),  (3l)  will  prove  worthwhile.  A more  tract- 
able alternative  might  be  to  drop  the  integer  restrictions  (29)  and 
solve  the  linear  program  (26)-(28)  instead.  In  this  case,  the  upper 
and  lower  bounds  b. .,  b. . are  given,  respectively,  by  the  greatest 
integer  not  exceeding  the  maximal  objective  value  obtained  in  (26)-(28), 
and  the  least  integer  not  less  than  the  minimal  objective  value. 

A further  relaxation  of  (26)-(28)  which  may  be  useful  can  be  obtained 
by  taking  a weighted  nonnegative  sum  of  the  constraints  (27)  to  transform 
(26)-(28)  into  a knapsack  problem: 


minimize 


subject  to 


Here,  \ > 0 is  an  m. -vector  of  weights  which  may  be  chosen  in  any 


reasonable  manner,  and  we  shall  not  elaborate  on  this  point.  (The  choice 


in  which  case  the  functional  constraint  in  problem 


(32)  is  simply  the  sum  of  the  constraints  (27)  - will  usually  suffice.) 


Since  linear  programming  knapsack  problems  can  be  solved  almost  by 


inspection,—  the  bounds  b..,  b . . obtained  from  (32)  require  little 

1 J 1 J 

more  computational  effort  than  those  provided  by  (2l) , (22),  and  will 
generally  be  significantly  tighter. 

We  conclude  this  section  with  a word  of  caution  concerning  one 
invalid  procedure  for  computing  upper  bounds  b_.  ..  on  the  allocation 
variables.  Consider  the  problem, 

maximize  c^^x''^'  , 


subject  to 


x^  > 0,  integer  , 


for  j = 1,  2,  ...  , N.  Problem  (33)  is  the  relaxation  of  (S.)  obtained 

* J 

by  assigning  arbitrarily  large  values  to  b^  , or,  equivalently,  by 

dropping  the  linking  constraints  entirely.  Denoting  the  (finite)  optimal 

solution  (if  one  exists)  to  (33)  by  xv^,  it  is  tempting  to  take 

b^  = A^x^  as  providing,  simultaneously,  upper  bounds  on  b.  . 

for  all  i = 1,  2,  ...  , m.  The  reasoning  is  that  given  (in  effect) 

th 

infinite  amounts  of  the  linking  resources,  the  j subproblem  uses 
only  A^x^  units  of  these  resources  in  an  optimal  solution.  Hence, 

. V & 

there  must  exist  an  optimal  allocation  b'J  , j = 1,  2,  . . . , N,  of 
the  (finite)  vector  of  linking  resources  b,  with  b^  < A^x^ 
for  each  j = 1,  2,  ...  , N. 

Unfortunately,  this  line  of  reasoning  is  erroneous,  and  it  is  easy 
to  construct  counter  examples.  The  difficulty  arises  from  what  might 


— The  solution  of  bounded-variable  linear  programming  knapsack  problems 
is  discussed  in  Chapter  k. 
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be  called  a substitution  effect  among  the  linking  resources.  Thus, 


units  of  the  i linking  resource  are  used 


while  only 


when  unlimited  amounts  of  all  m linking  resources  are  available',  an 


can  cause 


increased  consumption  of  the  other  resources  in  the  corresponding  optimal 


subproblem.  In  fact,  the  substitution  of  other 


solution  to  the 


resources  as  an  adjustment  to  the  decreased  availability  of  a given 


scarce  resource  is  a common  phenomenon  in  mathematical  programming 


Of  course,  when  (P)  has  only  a single  linking  constraint  the  sub 


stitution  effect  is  eliminated.  In  this  case,  the  upper  bound  b 


obtained  from  (33)  is  indeed  valid 


CHAPTER  3 

BRANCHING  IN  THE  RUDIMENTARY  BRANCH -AND- BOUND  ALGORITHM 


The  branching  step  (step  5)  in  the  Rudimentary  Branch-and-Bound 

Algorithm  (RBBA)  presented  in  Chapter  2 is  incomplete  insofar  as  no 

decision  rules  have  been  given  to  guide  the  selection  of  the  branching 

variable  b.  . or  the  specification  of  the  revised  bounds  associated 
1oJo 

with  the  partitioning  process.  Before  addressing  these  issues,  however, 
it  will  be  useful  to  introduce  some  preliminary  concepts  and  notation. 


3. 1 Preliminaries 

We  begin  with  a key  definition. 


Definition:  An  allocation  b.  . = b , 1 < i < m,  1 < j < N,  is 

feasible  if  (b^  , ...  , b^  ) e F^,  where  b^ 

= ( b y • • • y b^. ) p j = 1)  •••  j N j a nd.  Fq  is  as 

given  by  equation  (2.16). 


Note  that  the  feasibility  of  an  allocation  (b^1'*  , ...  , b^  ) is 

defined  in  terms  of  membership  in  the  set  F , without  regard  to  whether 

the  subproblems  (S.)  are  feasible  under  (b^  , ...  , b^  ). 

J * 

Next,  let  v(S.|b^  ) denote  the  maximal  objective  value  in  the 
J 

tb 

j subproblem  (s.)  when  the  linking  constraint  right-hand  sides  in  (S.) 

•¥■  J 

( 1 ) ( j ) / * * N 

are  set  to  bVtJ'  = bVJ;  = (b...,  ...  , b .).  Similarly,  let 
v(p|(b^,  ...  , b^  ))  denote  the  maximal  objective  value  in  problem 
(P)  under  the  allocation  b„  = b„,  l<i<m,  1 < j < N.  That  is, 

v(p|(b(l)  , ...  , b(N)  ))  = £ v(S  |b(j)  ). 

j=l  J 
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Definition:  A feasible  allocation  (b 


is  dominated 


if  there  exists  another  distinct  feasible  allocation 


Also,  a set  S of  feasible  allocations  will  be  called  dominated  if  every 
allocation  (b^1  ',  ...  , b^)  e S is  dominated.  Clearly,  any  allocation 


or  set  of  allocations  which  can  be  shown  to  be  dominated  will  not  have 


to  be  examined  further  in  the  search  for  an  optimal  allocation 


Finally,  from  the  definitions  of  the  sets  F , F. , F, 


Chapter  2 (see  equations  (2.16),  (2.17),  (2.l8)},  it  is  seen  that  each 


of  the  subsets  F differs  from  the  original  set  of  feasible  allocations 


and  other  subsets  F. 


n,  only  in  the  values  of  the  bounds 


on  the  allocation  variables.  Suppressing  the  common  require 


ments 


F may  be  identified  by  these  bounds.  Thus,  in  the  following  discussion. 


we  adopt  the  notation  shown  in  Figure  1 for  representing  the  nodes  in 


the  search  tree.  The  node  depicted  in  Figure  1 would  correspond  to  the 


3.2  Branching  in  the  RBBA 


The  branching  rule  developed  in  this  section  is  motivated  primarily 


by  the  desire  to  keep  as  small  as  possible  the  number  of  allocations 


which  must  be  examined  overall  in  the  course  of  the  search.  This  objective 


is  accomplished  by  first  recognizing  that  non-zero  slacks  in  the  optimal 


subproblem  solutions  can  be  used  to  identify  dominated  allocations,  and 


Node  n 


Figure  1:  Representation  of  node  n in  the  search  tree 


then  branching  in  such  a way  as  to  eliminate  these  allocations  from 


further  consideration.  The  procedure  is  derived  from  the  following 


simple  results 


Theorem  1:  Consider  the  block  angular  problem  (p),  and  let 


) be  a feasible  allocation.  Suppose 


subproblem  (S.  ) is  solved  using  the  vector  of  linking 


Assuming  (S..  ) is 


constraint  right-hand  sides  b 


feasible  under  the  allocation  b 


denote  the  optimal  solution,  and  s = 

* \ 

• . , s . ) the  corresponding  vector  of  slacks  in 


-,vl_ ....  „ . ^ 


the  linking  constraints.  That  is, 


(JQ>*  (J0)*  (J0)  (JJ* 

su  =bu  - A U x u >0. 


Then  if  any  s. . >0  for  some  i = 1,  2,  ...  , m,  the 

1Jo 

. . # f 

allocation  (b'  , . . . , tr  ) is  dominated. 


Proof:  Let  j , index  some  other  subproblem  (S.,  ),  where  1 < .1,  < N, 

Jq  — 1 — 

j..  4 and  consider  the  allocation  (b^  , ...  , b^  ) defined  by 

-¥•  * 

, (O/  (J0) 

b + s , for  j = I jQ 

(j)’  (J0^  (J0^ 

=P  -s  0 , for  (1) 


^ ^ <3  “ ,jN  y j ^ jQ,  Ji  • 

v*  . .* 

We  will  show  that  the  allocation  (V  , ...  , b^W'  ) is  dominated 

by  (b^1^  , ...  , b^  ). 


Firstly,  under  the  hypothesis  that  s >0  for  some 

;° 

i = 1,  2,  ...  , m,  it  is  clear  that  (b^  , ...  , b^  ), 

n ) t (n)’\ 

(bv  , ...  , bv  ) are  distinct.  Hence,  consider  the  problem, 

(d0)  (V 

(S!  ) maximize  c x , 

J0 


subject  to 


(V  (V 

A u x u < b u 


B(J°)x(J0)  <P<J°’ 

(j0} 

x > 0,  integer  . 

u0)’  (V* 

Since  b < b , the  set  of  feasible  solutions  to  (S’.  ) is 

J0 


a subset  of  the  set  of  feasible  solutions  to  (S.  ).  However,  since 


(V  (V  (J0}  ' (*30) 


A x = b 


is  still  feasible  for 


(S!  ),  and  hence  must  also  be  optimal  for  (s!  ). 

J0  M (a  \*  J0  , . 

( J ^ ) (jJ 

Thus,  since  b > b , and  since  x is  optimal  for 


both  (S.  ) and  (S’.  ),  it  follows  from  (l)  that 
JC  J0 


v(Sj|vj;  ) , for  3 - 1,2,...,n,  j t J0,JX 


v(Sj!bvJJ  ) j = v(s^  |b  ),  for  j = jQ 


, (Jx)* 

5 V(S,  lb  )>  for  j = j,  • 
J1  x 


N Ml*  N O' 

Summing  over  j in  (2),  £ v(s.jbu;  ) < £ v(S.|bU;  ) , or,  equiva- 

0=1  J 0=1  J 


lently , v(p|(b^  , ...  , b1'^  ))  < v(p|(b^',  ...  , b^')). 


It  follows  by  definition  that  the  allocation  (b'X  , , VN'  ) 


is  dominated. 


Q.E.D. 


Corollary.  Under  the  same  hypotheses  as  in  Theorem  1,  and  with  the  allo- 


cation (b^  , ...  , 


defined  by  (l),  the  set  of 


allocations 


(J„)’  (J„>  (J„f  u„>  . (J„) 


SS((bW,  ...  , b»)  e F |b  ^ <b  °'<b  ) 


is  dominated. 


Proof:  Arguing  as  in  Theorem  1,  each  allocation  (b^,  ...  , b^)  e S 


can  be  shown  to  be  dominated  by  the  allocation  (b^  , ...  , b^*). 


4l 


Q.E.D. 


r 


; 


For  completeness,  it  is  also  necessary  to  consider  the  case  where 

one  of  the  subproblems  (S.)  is  discovered  to  be  infeasible  under  a given 
* J * 

allocation  (b^  , ...  , b^  ). 

.*  * 

Theorem  2 : Consider  the  allocation  (b^  , ...  , b ) e FQ,  and 

suppose  the  subproblem  (S.  ) is  found  to  be  infeasible  when 

J0 

solved  using  the  vector  of  linking  constraint  right-hand 

(J0>  ClI* 

sides  b . Then  (bv  , ...  , b^  ) is  dominated. 
Proof:  Adopting  the  standard  convention  that  v(s  ,|b  ) = -«  if 

<dJ*  J 

(S.  ) is  infeasible  under  b , it  follows  that 
J / 


v(p}(b^  , ...  , b^  ))  = Hence  (b*1^  , ...  , b^  ) is  doai- 

f i) « (fj)  T 

nated  by  any  other  allocation  (bv  , ...  , bv  ) ■ FQ. 

Q.E.D. 

Corollary:  Under  the  same  hypotheses  as  in  Theorem  2,  the  set  of  allo- 

cations 


.(N) 


(1) 


.(N) 


S 5 «b(1),  ...  , b<»>)  £f0lb(;lo)  <b(J°’  ) 


(M 


is  dominated. 


■ 


■ I 


Proof:  For  each  allocation  (b^1;,  ...  , b^)  e S,  (S.  ) must  also 

Jo 


(«U 


be  infeasible  under  b 


The  result  follows  by  Theorem  2. 


Q.E.D. 


Theorems  1,  2,  and  their  corollaries  are  the  main  results  on  which 
the  branching  rule  is  based.  Specifically,  we  wish  to  branch  in  such  a 
way  as  to  rule  out  of  further  consideration  any  set  of  allocations  S 


1+2 


satisfying  either  (5)  or  (4).  Suppose  at  a given  node  n in  the  RBBA, 

the  allocation  (b^  , ...  , ) e F is  selected  to  be  examined 

n „ 

(i0) 

next,  and  subproblem  (S,  ) is  solved  using  b . For  i = 1,2,..., m, 


* _0  (jJ  (jJ  (j  ) 

b - h A.,  x,  , if  (S.  ) is  feasible  under  b 
1J0  k=l  1K  K J0 


, if  (-S  . ) is  infeasible  under  b , 
J0 


where  x 


is  as  in  Theorem  1 and  M is  an  arbitrarily  large  number. 


Then  using  the  notation  in  Figure  1,  the  branching  at  node  n is  done 
as  indicated  by  Figure  2.  The  bounds  b. . , b , i = 1,  2,  ...  , m, 

. 1J0  1Jo 

in  subproblem  (S.  ) at  node  n+1  are  revised  as  a result  of  the  branching 
J0 

procedure.  Each  of  the  right-hand  branches  added  to  the  search  tree  in 

Figure  2 defines  a new  subset  F,  to  be  added  to  the  list  and  examined 

K 

later.  However,  the  subset  Fn+^  represented  by  node  n+1  is  considered 


The  new  bounds  on  the  allocation  variables  b.  . , i = 1,2-,  ...,m, 

1J0 


in  (S . ) at  node  n+1  are  given  by 


b.  . = b.  . - s.  . and  b.  . = b.  . , for  i =.  1,  2,  . . . , m , (6) 

—1  1 I I IT  IT  1 1 ' 77  7 7 \ ' 


"1J0  1J0 


1Jo  1Jo 


where  s. . , i = 1,  2,  ...  , m,  is  given  by  (5).  If  (S.  ) was  feasible 

>0  <j0>* 

under  the  vector  of  linking  constraint  right-hand  sides  b , then 
b..  = b!  , i = 1,  2,  ...  , m,  where  b is  as  defined  in  (l). 

° J°  (J0)r  (V* 

In  the  set  Fn+^,  we  have  in  this  case  that  b < b < b , 
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and  it  follows  from  the  corollary  to  Theorem  2 that  every  allocation 


• (j’o}  (V  l/ 

in  Fn+-^  is  dominated,  except  those  with  b = b .—  Conse- 


quently, we  can  fix  the  allocation  variables  in  subproblem  (S^  ) at 

«o>  °° 

the  values  b = b at  node  n+1,  and  also  in  any  nodes 


descendant  from  node  n+1,  thereby  ruling  out  of  further  consideration 


the  set  of  allocations  S defined  by  (3).  Moreover,  since,  as  seen 

cu* 


in  Theorem  1,  the  optimal  solution  x v obtained  in  (S.  ) under 

eg*  (V  J° 

the  allocation  b remains  optimal  under  b , it  will  not 


be  necessary  to  resolve  the  j subproblem  at  node  n+1,  nor  at  any 


of  its  descendants. 


Similar  remarks  apply  if  (S.  ) was  infeasible  under  the  vector 


of  linking  constraint  right-hand  sides  b . In  this  case,  it  is 


sufficient  in  (5)  to  choose 


M > max 


1 < i < m 


(b.  . - b.  . ) , 

' 11  — i n ' 


where  here  b..  is  the  initial  lower  bound  on  b. . , i = 1,2,..., m, 


used  in  defining  the  set  F . Then  after  branching  as  in  Figure  2, 


the  set  F . corresponding  to  node  n+1  consists  of  all  allocations 

1 1^  f (<U  ( JfJ 

(bv  , ...  , bv  ) e Ffi  with  b < b .By  the  corollary  to 


Theorem  2,  all  these  allocations  are  dominated^  hence,  F is  fathomed. 


In  this  way,  the  desired  result  is  achieved:  the  set  S of  dominated 


allocations  defined  in  (b)  is  ruled  out  of  further  consideration. 


Note  that  if b! , < b . for  some  i = 1,  2,  ...  , m (where  here 
1J0  1J0 


b. . is  the  lower  bound  on  b. . in  the  initial  set  F ),  the  set  F , 
-iJ0  id0  n"  n+1 

consists  entirely  of  dominated  allocations.  In  this  case,  node  n+1  is 


immediately  fathomed. 


The  branching  procedure  is  clarified  with  the  aid  of  a small  example. 
Consider  the  problem,— ^ 


maximize  8x^  + 5x2  + 6x^  + 9 \ + 7x^  + 9xg  + 6x^  + 5xg  , 


subject  to 


5xl  + 5x2 


+ 3xg  + 4x^,  + 6xg  < 15 


+ lot,  + 5x,  + 7xc 


2xl  + ^x2  + ^x5 

7xi  + 3x2  + 6x3 


3x4  + x5  + 2x6 
2xh  + 4x5  + 3x6 


8x^  + 5Xg  < 25 
7x?  + 9xg  < 30 
6x^  + 4xg  < 20 


x^  > 0,  integer,  j = 1,  2,  ...  , 8 . 


Problem  (8)  is  a block  angular  integer  program  of  the  form  (P),  with 

m = 2 linking  constraints  and  N = 3 subproblems.  Using  equations 

(24),  (23)  of  Chapter  2,  the  initial  bounds  0 < b^  < 15,  0 < b^  < 10, 

j = 1,  2,  3>  can  be  derived  to  define  the  feasible  set  F • 

.*  * # 0 

Suppose  the  initial  allocation  (b  1 ; b^2  ) b'^  ) - (2,10; 

9,0;  4,0)  is  selected  for  examination  at  node  0.  Problem  (8)  decom- 
poses, and  the  first  subproblem  becomes 


— This  example  is  taken  from  Hillier  and  Lieberman  (1974),  p.  145.  The 
right-hand  sides  in  the  linking  constraints  have  been  halved,  and  integer 
restrictions  imposed  on  x.,  j = 1,  2,  ...  , 8. 


i 


maximize 


8x^  + 5x2  + 6xj 


> 


subject  to 


5xx  + 3x2  <2 

2x1  + 4x^  < 10 

2x1  + hx^  + 3Xj  < 10 

7X-L  + 3x2  + 6x^  < 6 

5x1  + 3xj  < 12 

x.  > 0,  integer,  j = 1,2,3 
1 


* * *. 


The  optimal  solution  to  (S^)  is  easily  seen  to  be  x'  = (x^,x2,x^) 

. . (l)*  , * * x 

= (0,0,2),  so  that  in  this  case  the  slack  vector  is  s'  = (s1^>s2i' 

= (2,2).  By  the  corollary  to  Theorem  1,  the  set  of  allocations 

S = {(b(l),b(2),b(5))l(0,8)  < (bn,b21)  < (2,10),  (bn,b21)  4 (0,8)) 

is  immediately  seen  to  be  dominated,  and  the  partitioning  of  Fq  is 


carried  out  as  shown  in  Figure  3«  At  node  1 in  Figure  3,  and  also  in 
any  of  its  descendant  nodes,  the  allocation  variables  in  the  first  sub- 
problem may  be  fixed  at  b-)  ■)  = 0,  = 8,  since  all  other  allocations 

in  the  set  F,  represented  by  node  1 are  dominated.  As  in  Theorem  1, 

* ^ 

x^  = (0,0,2)  remains  optimal  for  the  first  subproblem  in  the  case 
b^  = (0,8) , and  consequently,  subproblem  1 will  not  need  to  be 
resolved  at  any  nodes  descendant  from  node  1. 


3.3  Additional  Comments  on  the  Branching  Procedure 


The  above  example  also  serves  to  illustrate  that  often  it  will  not 


be  necessary  to  add  to  the  search  tree  all  of  the  branches  indicated  in 
Figure  2.  The  general  branching  procedure  generates  two  new  subsets 
for  each  allocation  variable  b , i = 1,  2,  ...  , m,  in  the  subproblem 


kj 


Figure  3:  An  example  of  the  branching  process 


(S.  ) which  has  Just  been  solved  using  the  allocation  b } for 
J0 

fixed  i = i , F , say,  corresponds  to  the  set  of  allocations  with 

o 

♦ 

b.  > b.  +1,  and  F corresponds  to  the  set  of  allocations  with 


b.  . < b!  . - 1 = b.  . - s.  . -1.  However,  if  b.  . = b.  . 

Vo~  Vo  Vo  Vo  Vo  Vo 

(where  b.  . represents  the  upper  bound  on  b.  . in  the  initial 

Vo  Vo 

set  F ),  the  set  F is  obviously  empty,  since  in  this  case  b.  . 
n n±  Vo 

« * — * 
would  be  required  to  satisfy  b.  . + 1 < b . < b.  . = b.  . . 

Vo  Vo  ~ Vo  Vo 

& * 

Similarly,  if  b.'  . = b.  . - s.  . = b.  . , the  set  F is  empty. 

Vo  ~ Vo  Vo  Vo  2 

In  either  case,  the  corresponding  branch  in  Figure  2 is  omitted.  In 


■7T  VT 

terms  of  the  example,  note  that  neither  the  branch  b^  < b^  1 - s^  - 1 
(=  -l)  nor  the  branch  bg^  > bg^  +1  (=11)  is  included  in  Figure  3. 

f l) • 

It  should  be  noted  that  the  allocation  (bv  , ...  , bv  ) defined 
by  (l)  is  used  only  to  facilitate  the  proof  of  Theorem  1.  In  general, 

(JJ* 


the  slacks  s 


may  be  re-allocated  among  several  other  subproblems 


in  a more  promising  manner  than  in  (l).  This  point  is  developed  in 
Chapters  4 and  5. 

(j) t (N) * 

In  any  event,  the  definition  of  the  allocation  (bv  bv  7 ) 

in  Theorem  1 presupposes  the  existence  of  another  subproblem  (S.  ) to 


which  the  unused  resources  s 


may  be  re-allocated.  By  the  time 


the  subproblem  (SN)  is  solved,  however,  the  allocations  to  the  previous 
N-l  subproblems  have  already  been  fixed  in  accordance  with  the  branching 
procedure  described  above,  and  cannot  be  revised.  This  difficulty  is 
resolved  by  noting  that  once  the  allocations  b^,  j = 1,  2,  ...  , N-l 
are  fixed,  there  is  only  one  feasible  allocation  of  interest  in  (SN), 


namely, 


b . "j1  .,<■*> 


Any  allocation  which  uses  less  than  all  of  the  remaining  linking  resources 


W 
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in  (SN)  is  dominated  by  the  allocation  in  (9).  Consequently,  no  branching 

"fch 

is  required  following  the  solution  of  the  N 1 subproblem  in  Step  4 

of  the  RBM.  Step  5 is  bypassed,  and,  jumping  to  Step  2 instead,  a new 

subset  F,  is  selected  for  examination  next, 
k 

The  advantages  of  adopting  the  branching  procedure  described  in 

Section  3-2  are  several.  Firstly,  the  procedure  enables  the  search  tree 

to  be  proved  substantially  (and  quickly)  by  eliminating  searches  over 

the  dominated  sets  S as  defined  in  (3)  or  (4).  This,  of  course,  has 

been  the  primary  motivating  force  in  its  development.  Secondly,  the 

procedure  provides  an  easy  decision  rule  for  use  in  Step  5 in  the  RBBA 

presented  in  Chapter  2.  The  choice  of  the  branching  variables  and  the 

specification  of  the  bounds  used  in  the  partitioning  process  result 

directly  from  the  subproblem  solutions  obtained  in  Step  4,  with  only 

minor  additional  computational  effort  required.  Thirdly,  in  general 

only  one  subproblem  (S.)  will  be  solved  in  Step  4 in  the  RBBA  before 

J 

the  given  allocation  is  recognized  to  be  dominated.  Jumping  then  to 

(j)* 

Step  5 and  branching  as  described  above,  the  unused  resources  s 
discovered  in  solving  (S.)  can  be  reallocated  to  the  remaining  subproblems, 
thereby  possibly  yielding  an  improved  allocation  overall.  Finally,  the 
branching  procedure  allows  the  objective  function  bounds  computed  at  sub- 


sequent nodes  to  be  tightened,  and  thereby  can  aid  in  fathoming  in  the 
RBBA.  Since  in  the  resultant  node  n+1  in  Figure  2 the  allocation  to 

(d0)  (jn)’  (jJ*  (iJ* 

subproblem  (S.  ) is  fixed  atb  = b = b -s  , the 

J0 

exact  (integer  programming)  objective  value  obtained  in  (s_.  ) can  be 

J0 

used  when  computing  the  upper  bound  U on  the  objective  function  value 
in  (P)  in  Step  3-  This  of  course  holds  for  all  nodes  descendant  from 
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node  n+1  as  well,  since  the  allocation  to  the 


subproblem  remains 


fixed  at  these  nodes.  These  last  remarks  are  clarified  in  Chapters  4 


and  5,  where  the  bounding  procedures  are  discussed 


CHAPTER  b 


COMPUTING  BOUNDS:  SINGLE  LINKING  CONSTRAINT  CASE 


In  this  chapter  we  are  concerned  with  Step  3 of  the  RBBA  presented 


in  Chapter  2,  i.e.,  the  problem  of  computing  an  upper  bound  U on  the 


maximal  objective  value  attainable  in  (P)  under  any  allocation 


To  this  end,  a Master  Problem  associated  with  (P)  is  introduced  in 
Section  4.1.  The  solution  to  this  problem  provides  the  desired  bound 


U.  In  addition,  in  the  case  that  the  upper  bound  U does  not  fathom 


the  solution  of  the  Master  Problem  generates  a promising  allocation 


as  required  in  the  second  part  of  Step  3 - which  may  be  selected  next 


for  further  examination  in  Step  4 


When  the  original  block  angular  problem  (p)  has  only  a single  linking 


constraint  (as  might  be  the  case,  for  example,  for  a multidivisional 


corporation  in  which  the  subdivisions  are  tied  together  only  by  a cor 
porate-wide  capital  budgeting  limitation),  the  Master  Problem  itself 


takes  on  a special  structure  which  gives  rise  to  a specialized  and  very 


efficient  solution  procedure 


bounding  procedure  in  this  chapter  is  devoted  entirely  to  the  single 


linking  constraint  case.  Although  the  fundamental  concepts  involved 


in  generating  the  objective  function  bounds  and  trial  allocations  in  the 


multiple  linking  constraints  case  are  quite  similar  to  those  developed 


— In  fact,  in  this  case  the  Master  Problem  is  seen  to  belong  to  the 
broader  class  of  concave-separable  integer  programs  discussed  in  Kochman 
(1976). 


here,  discussion  of  the  details  of  the  bounding  procedure  when  there  is 
more  than  one  linking  constraint  is  deferred  to  Chapter  5. 

4.1  The  Master  Problem 

The  transformed  block  angular  problem  (P1)  given  by  (2.7)-(2.lo) 
might  itself  be  rewritten  in  the  form, 


maximize 


subject  to 


N / . \ 

0=1  J 


JV  ( . \ 

i b(j)  < b 

0=1 

^(j)  <b(d)  <b(d)^  b(d)  integer  for  j 


(1) 


- 1,2, . . . ,N  , 


wheie  for  each  j = 1,  2,  ...  , N,  b^  = (b-^,  •••  > ) , b^  = 

(b..,  ...  , b .),  and  f.(b^)  is  the  maximal  objective  function 
-4j  J 

value  in  subproblem  (S.),  as  a function  of  the  vector  b^  of  linking 

J 

resources  allocated  to  (S.).  That  is, 

v 0 


f.(b^)  = max  c^x^/>  , 

J ’ 


subject  to 


A(d)x(d)  < b(j) 
B(d)x(J)  < p(d) 

x(j)  > 0,  integer  , 


(2) 


for  j = 1,  2,  . . . , N. 

From  the  definition  of  the  set  F^  of  feasible  allocations  in 
(2.16),  it  is  apparent  that  problem  (l)  can  be  more  compactly  expressed 
as 
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s.-vy 


maximize 


(b(l),...,b(W))eFr 


N / . \ 

0=1  J 


Moreover,  as  discussed  in  Section  3.1,  the  subsets  of  F^  generated 

in  the  course  of  the  RBBA  differ  from  each  other  only  in  the  values  of 

the  upper  and  lower  bounds  b. b. . on  the  allocation  variables  b, . , 

ij  -ij  ij 

1 < i < m,  1 < j < N.  Consequently,  the  problem  of  determining  the 
maximum  objective  value  attainable  in  (P)  by  any  allocation  in  a given 
subset  F^  can  be  represented  by 

N , 

maximize  S\  f.(b'J')  (4) 

(b(1>„..)b('l>)cFn  i-1  ° 

by  revising  the  bounds  b^,  b^,  j = 1,  2,  ...  , N,  in  (l)  appropri- 


ately. 


In  order  to  derive  an  upper  bound  U on  the  optimal  objective  valm 


in  (U),  consider  the  relaxation  of  (b)  obtained  by  replacing  the  objective 

function  components  f.(b.),— ^ j = 1,  2,  ...  , N,  with  their  linear- 

J 1 


programming  approximations.  That  is,  for  j = 1,  2, 


, N,  let 


z . (b  . ) = max 
3 3 ~ 


subject  to 


c(j)x(l) 


A(j)x(l)  < b. 

- J 

> 0 , 


— When  the  block. angular  problem  (P)  has  only  a single  linking  constraint, 
the  matrices  A contain  only  one  row,  and  the  vector  of  allocation 
variables  b^  reduces  to  a scalar.  In  identifying  the  individual  allo- 
cation variables  b.  , we  therefore  suppress  the  first  subscript,  and 
write  b ^ = (b..)  = b.  in  the  following  discussion. 

•tf  J 


and  consider  the  problem, 


(MP) 


maximize 


N 

£ 

j=i 


z .(b.) 

y j 


y 


subject  to 


N 

£ b < b 
0=1  J 


b.  < b.  < b.,  b. 

-o-o-o  0 


integer,  for  j 


1,2 


; • 


,N  . 


Insofar  as  problem  (MP)  serves  to  coordinate  the  activities  of  the  sub- 
problems, we  refer  to  (MP)  as  the  Master  Problem,  borrowing  loosely  from 
the  terminology  of  Dantzig  and  Wolfe  (i960).  It  is  clear,  since  (5)  is 
a relaxation  of  (2),  that  z.(b.)  > f.(b.)  for  all  b.  < b.  < b., 
j = 1,  2,  ...  , N,  and  so  the  maximal  objective  value  in  (MP)  indeed 
generates  an  upper  bound  for  the  objective  value  in  (4). 


4.2  Solving  the  Master  Problem 

In  the  presence  of  a single  linking  constraint,  problem  (5)  may  be 

considered  as  a parametric  linear  program,  and  can  be  solved  by  standard 

linear  programming  techniques.  It  is  well-known  that  the  resulting 

functions  z.(*)  defined  in  (5)  will  be  concave  and  piecewise  linear; 

0 

that  is,  each  z.(b.),  j = 1,  2,  ...  , N,  has  the  form  shown  in  Figure  1. 

0 0 

As  such,  and  because  of  the  integer  restrictions  on  the  allocation  vari- 
ables b.,  j = 1,  2,  ...  , N,  (MP)  is  a concave  separable  integer  program. 
J 

This  class  of  problems  is  defined  and  discussed  in  greater  generality 
in  Kochman  (1976).  The  algorithm  developed  in  this  section  for  solving 
the  problem  (MP)  is  a specialized  version  of  the  procedures  presented 


there. 


Figure  1:  Form  of  the  concave  and  piecewise  linear  function  z 


Ignoring  the  integer  restrictions  on  the  b 


for  the  moment,  (MP)  becomes  a concave-separable  linear  program  in  bounded 


variables.  As  shown  in  Figure  2,  let  p 


denote  the  k 


breakpoint 


, including  the  lower  and  upper  bounds  b 


denote  the  number  of  such  breakpoints,  so  that 


Without  loss  of  generality,  we  may  assume  that  p 


valued 


denote  the  slope  of  the  k 


segment  of  z 


the  intercept.  Then,  for  j = 1,2 


may  be  written  as 


yj2— I 

I ' 


yj5" 


I'-  y, 


' i 


Figure  2:  Breakpoints  p and  auxiliary  variables  y 

J J K 

associated  with  z.(b.) 

J J 


W ‘ Vj  + V f0r  Pjk  i bJ  < PJk+1-  . (7) 

To  linearize  (MP),  the  auxiliary  variables  y (k  = 1,  2,  ...  , K.-l), 

Jk  j 

j = 1,  2,  ...  , N)  are  introduced  as  indicated  in  Figure  2.  We  can 
then  write 


K.-l 


b . = p . 

J Jl 


n + £ 


k=l 


yjk> 


j - 1,  2,  ...  , N ) 


(8) 


where  each  y.,  is  bounded  by 


0 1 Fjk  1 pjk+l  Pjk’  k - 1,2, . . . ,K  , J - 1,2, ... ,N 


(9) 
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• ■j.'At  j is 


Also,  for  j = 1,  2, 


, N, 


V1 


z . = 


SjlPjl  " “ol  + k^  Sjkyjk 


(10) 


It  is  easy  to  see  that  (10)  gives  the  same  results  as  (7)  whenever 

and  y (k  = 1,  2,  ...  , K . - l)  are  related  as  in  (8),  (9),  provided 
Jk  J 

that  no  y,k  > 0 unless  yJk_1  = pJk  - Pjk_1J  i-e.,  unless  y^^  is 
at  its  upper  bound. 

Still  ignoring  the  integer  restrictions  on  the  allocation  variables 

b.  (j  = 1,  2,  ...  , N)  and  using  (8),  (9),  (10),  (MP)  becomes 
J 


N 


K,-l 


2 + 


0=1 


max 


subject  to 


£ £ 

0=1  k=l 


N Kj'X 


Sjkyjk  ’ 


K D yjk  * » - S p 


(id 


j-1  k=l 


0=1 


jl 


° S y3k  i pdktl  - v 3 ' i’2> ' ' ' '*>  k ’ i’2'- • ’ Y 1 • 

It  is  a standard  result  of  the  theory  of  concave-separable  linear  programs 
that  the  optimal  solution  to  (ll)  will  naturally  satisfy  the  condition 
that  no  y^k  > 0 unless  y^k  ^ = p^k  - p..k  Thus,  (U)  is  equivalent 
to  the  LP  relaxation  of  the  Master  Problem  (MP) . 

Note  that  (11)  itself  is  a linear  programming,  bounded- variable 
knapsack  problem,  and  so  its  solution  is  almost  trivial.  The  general 
LP  bounded-variable  knapsack  problem  is  of  the  form, 
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maximize 


t > 


subject  to 


5)  a .x.  < b 

LJ  i i — 


0 < x . < x . , 

- j - j 


where,  without  loss  of  generality,  c . > 0,  a . > 0,  j = 1,2, . . . ,n.— 

J J 

The  solution  to  (12)  is  obtained  by  first  computing  the  ratios 

c./a.,  j = 1,  2,  ...  , n,  and  reordering  the  variables  x.  so  that 
J J J 


c.  /a.  > c.  /a.  > •••  > c.  /a, 

J1  J2  ~ ^n  Jn 


Then,  starting  with  x.  and  proceeding  sequentially  to  x.  , x.  , 

J i ^2^3 

etc..,  each  variable  x.  is  set  to  its  upper  bound  x.  until,  for 

Ji  Ji 

some  index  p, 


^ _ * — 
Js  a x < b,  but  £ a x 

L=1  Ji  Ji  i=l  Ji 


Next,  x.  is  set  to  (b  - 5}  a.  x.  )/a.  , and  x.  is  set  to  zero 

Jp  i=l  ^i^i  Jp  ^i 

for  i = p+1,  p+2,  ...  , n.  Thus,  if  p is  that  index  for  which  (l4) 

holds,  the  optimal  solution  to  (12)  is 


— If  any  c > 0 and  a < 0,  it  is  always  optimal  to  set  x = x. , 

K K K.  K 

thereby  eliminating  x from  the  problem.  Similarly,  if  any  c,  < 0 
and  a^  > 0,  set  x^  = 0.  Finally,  if  any  < 0 and  ak  < 0,  make 
the  transformation  x ’ = x - x . Substituting  x'  for  x in  (12) 

K K K K.  K 

then  renders  c,'  > 0,  a,'  > 0. 

k — k 
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| -rTnTiTr^miirnTTini|i  nimiiii— mwjMliilMWi  i 


1 < i < p-1 


Xji  = ao. 


JT  

(b  - 3 x ),  i = P 

i=l  Ji 


, p+1  < i < n , 


where  the  indexing  sequence  j is  defined  by  (13). 

Problem  (ll)  is  a special  case  of  problem  (12)  in  which  each  con- 
straint coefficient  is  identically  equal  to  1.  Thus,  the  sequence  j^ 
in  (13)  in  this  case  can  be  obtained  simply  by  reordering  the  s..  , 

JK 

1 < j < N,  1 < k < K,  - 1,  in  non-increasing  order  since  no  divisions 

J 

are  necessary* 

If  (ll)  is  solved  by  the  procedure  outlined  above,  it  is  clear  from 

(15)  that  at  most  one  y , say  y , will  not  be  at  its  upper  or  lower 

jk  rp 

jf, 

bound  in  the  optimal  solution  y'  = (y  ),  1 < j < N,  1 < k < K.  - 1. 

JK  — ” J 

Hence,  if  the  upper  bound  (9)  on  each  y.,  is  integer,  each  y 

JK 

(jk  ^ rp)  will  be  integer.  But  then  the  single  functional  constraint 
of  (ll)  shows  that 

„ K.-l 

* N N J * 

yrp  * b - Z P11  - jg  £ y3k  " inteeer  • 

jk/rp 

Therefore,  when  each  upper  bound  p.,  . - p in  (9)  is  integral, 

JK+i.  JK 

it  follows  that  there  exists  an  optimal  LP  solution  to  (ll)  which  is 

naturally  integer.  By  the  equivalence  between  (ll)  and  the  Master 

Problem  (MP),  this  solution  yields  the  optimal  LP  solution  to  (MP)  via 

(8).  Furthermore,  from  (8)  and  the  fact  that  each  y , j = 1,2,...,N, 

jk 

k = 1,  2,  . . . , Kj  - 1,  is  naturally  integer  in  the  optimal  solution 
to  (11),  it  follows  that  each  b^ , j = 1,  2,  ...  , N,  will  also  be 
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integer-valued  at  the  LP  optimum  to  (MP).  The  discussion  above  has  thus 
established 


Theorem  1:  Whenever  p..,.,.,  - p..,.  is  integral  for  all  j = 1,  2,  ...  , N, 

— J K+-L  J K 

k = 1,  2,  . . . , K.  - 1,  the  optimal  integer  solution  to  the 
J 

Master  Problem  (MP)  can  be  obtained  from  the  optimal  linear 
programming  solution  to  the  bounded-variable  knapsack  problem 
(11). 

We  next  show  how  to  modify  the  parametric  objective  functions  z.(b.), 

J J 

j = 1,  2,  , N,  in  such  a way  as  to  obtain  an  equivalent  problem 

(MP')  to  (MP)  in  which  all  the  breakpoints  p.,  are  integral.  This  con- 
dition  obviously  implies  that  ” ^jk  lnleSral  for  aH 

j,  k,  and  so  the  solution  procedure  described  above  may  be  applied  to 
(MP'). 

The  procedure  for  modifying  z.(b.)  is  the  same  for  each 

J 1 

j = 1,  2,  ...  , N.  Hence,  consider  z . (b . ) for  some  arbitrary  but 

0 J 

fixed  j.  Recalling  that,  without  loss  of  generality,  p and  PjK 

J 

may  be  assumed  to  be  integral,  let  p. . (2  < £ < K.  - 1)  be  any  non- 

— — J 

integer  breakpoint  of  z.(*).  We  seek  to  remove  p.«  as  one  of  the 

J 

breakpoints  of  z.(-)  by  creating  two  new  breakpoints  at  [p.»]  = 
greatest  integer  < p.„,  and  < p..  > = least  integer  > p...  This  is 

— JJi  j*  — JJt 

accomplished  by  replacing  z.(b.)  for  [p..]  < b . < < p..  > with  the 

J J J ■*'  J J * 

straight  line  segment  from  the  point  ([p.J,  z.([p.J))  to  the  point 

J J * 

(<  p..  >,  z.(<  p >)),  as  shown  in  Figure  3.  It  is  easy  to  see  that 
j*  J J* 

this  modification  preserves  the  concavity  of  z.(-).  Equally  important 

J 

(for  reasons  that  will  be  seen  shortly),  the  value  of  z.(b.)  is  changed 

J J 

under  the  modification  procedure  only  between  integer  values  of  b.. 
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y-‘  proposed  modification 

i to  zi(V 


[V 


<pJi> 


Figure  3:  Modification  to  z.(b  ) to  eliminate  non-integer  breakpoints 

0 J 


The  slope  s and  intercept  a of  the  modified  segment  of  zj(*) 
are  easily  computed: 


z,(<  Pi£  >)  - MtP,.]) 

J < lit  > - p^  » - V' V> ' 


since  < p..  > - [p.„]  = 1 for  non-integer  p.  Also,  since  the  value 

of  z.(b.)  at  b = [p..]  is  unchanged, 

J 3 J 

“ ‘ V[ V’  ‘ s[V  ’ (17) 


: 

. 


Letting  z’(b.)  denote  the  function  which  results  from  modifying  z (b.) 


as  in  Figure  3,  we  have,  specifically, 


z . (b  . ) , 
3 3 

for 

Pjl  < »3  < tPj,] 

sb . + a, 
3 

for 

z . (b  . ) , 
3 3 

for 

<*3l><»3<  PjK, 

J 

where  z.(b.)  is  given  by  (7). 
cJ  J 

The  modified  function  z!(b.)  does  not  have  a breakpoint  at 

J J 

b.  = p . Instead,  the  integer  points  b.  = [p.„]  and  b.  = < p > 

3 3Z  3 3£  3 J* 

are  breakpoints  of  z\ (b.),  while  they  may  not  have  been  breakpoints 

of  the  original  function  z.(b.).  Usually,  therefore,  the  modification 

J J 

procedure  results  in  increasing  the  total  number  of  breakpoints  by  one 

for  each  non-integer  breakpoint  of  z.(b  ).  If,  however,  [p.»]  or 

J 3 

< p..  > or  both  are  natural  breakpoints  for  z.(b.),  or  if  the  interval 
J * J <3 

[p..]<b  <<p..,>  contains  more  than  one  breakpoint  of  z . (b  . ) (so 

J*  - j — J*  J 3 

that  all  are  eliminated  simultaneously  by  the  same  segment),  then  the 
number  of  breakpoints  can  remain  the  same  or  even  decrease  following 

the  modif ication. 

The  modification  procedure  is  the  same  for  each  non-integer  break- 
point p..  of  z.(b.).  The  resulting  function  z*(b.)  is  concave, 
3^33  33 

piecewise  linear,  and  has  all-integer  breakpoints.  Repeating  the  pro- 
cedure for  each  j = 1,  2,  ...  , N leads  to  the  problem, 

N 

(MPf ) maximize  z!(b.)  , 

3=1  3 J 


subject  to 


j-i  3 

bj  < b^  < b^,  b^  integer,  j = 1,2,...,N  . 
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Since  only  integer  solutions  to  (MP)  are  of  interest,  and  z’(b.) 

J J 

= z.(b.)  at  all  integer  values  of  b.,  b.  < b.  < b.,  j ='1,2,...,N, 

J j 0 J j ~ 3 

it  is  clear  that  (MP-)  and  (MP)  must  have  the  same  optimal  solutions. 

We  may  therefore  solve  (MP1)  in  place  of  (MP).  However,  since  all  break- 
points of  z'(b  ),  j = 1,  2,  ...  , N,  are  integer,  it  follows  by  the 
J J 

arguments  above  that  the  integer  restrictions  on  b , j = 1,  2,  ...  , N, 

J 

in  (MP')  are  redundant  and  may  be  dropped.  Moreover,  since  the  z'(b.), 

J J 

0 = 1,  2,  ...  , N,  remain  concave  and  piecewise  linear  under  the  modi- 
fication procedure,  (MP‘)  is  a concave  separable,  bounded-variable  LP 
knapsack  problem,  and  may  be  efficiently  solved  using  the  procedure 
described  above. 

In  practice,  it  is  not  necessary  to  introduce  the  auxiliary  variables 

y explicitly  in  solving  (MP*).  Because  each  constraint  coefficient 
J K 

in  (MP1)  is  unity,  the  LP  solution  procedure  for  problem  (ll)  reduces 
simply  to  sequentially  choosing  the  largest  (remaining)  s . and  setting 

jK 

the  corresponding  variable  y to  its  upper  bound.  But  since  each 

z'(b.)  is  concave,  j = 1,  2,  ...  , N,  we  have  for  each  j that 
J J 


sjk  - Sjk+1’ 


k = 1,  2,  ...  , K.  - 2 . 


(14) 


Thus,  in  terms  of  the  original  variables  b.,  j = 1,  2,  ...  , N,  the 
solution  procedure  amounts  to  searching  over  j for  that  (remaining) 
segment  k of  z.(*)  with  largest  slope  s . , and  increasing  the 

J jK 

corresponding  variable  b.  from  p to  p.,  . . That  is,  b.  is 

J J"  J 

increased  to  its  next  larger  (integer)  breakpoint  value.  When  such  an 

N 

increase  would  result  in  making  £)  b.  > b,  the  chosen  variable  b 

j*l  0 r 

is  set  to  b = b - J)  b.  instead,  and  the  procedure  is  terminated. 
r & r J 
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The  procedure  for  solving  the  Master  Problem  (MP)  in  the  case  that 
the  original  block  angular  problem  (P)  has  only  a single  linking  constraint 


is  now  presented  in  algorithmic  form. 


Algorithm  for  Solving  (MP) : Single  Linking  Constraint 


Step  0.  For  j = 1,  2,  ...  , N,  solve  subproblem  (S^)  parametrically 

J 

in  b . to  obtain  the  functions  z.(b.).  Modify  z.(b.)  as 
3 3 3 3 3 

discussed  above  to  eliminate  all  non-integer  breakpoints  of 

z . (b  . ) . 

0 3 

Step  1.  (initialization.)  For  j = 1,  2,  ...  , N: 

Set  b . = p . . = b . . 

J Jl  ~3 

Set  kj  = 1. 


. . * _ 

Step  2.  Find  an  r (1  < r < N'  such  that  b < b and  s , > s 

- ' _ __  j*  27  v’  Lr  


Vj  = 1,  2,  ...  , N such  that  b*  < b . . 

JO 


rk  - jk. 
r d j 


Step  J.  Reset  k 


k + 1. 
r 


Then,  reset  b = p , . 

r *rk 

r 


Step  k.  (Check  for  termination.) 


if- 

If  ^ b > b,  reset  b^  = b - b.  and  stop 


j=l  0 


3tr 


(b  . , j = 1,  2,  ...  , N,  is  optimal  for  (MP)), 
J 

Otherwise,  go  to  Step  2. 


In  Step  1 of  the  algorithm,  each  variable  b.  is  initially  set  to 

J 

its  lower  bound  b . . The  k,  (j  = 1,  2,  ...  , N)  are  used  to  index 
"v  J 

the  segment  of  z.(b.)  which  is  currently  of  interest.  In  the  search 
J J 


I 


1 
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for  the  largest  (remaining)  s in  Step  2,  we  know  by  (19)  that  if 

Jk 

* 

b.  = p.v  , j a 1,  2,  , N,  then  only  the  slope  s.,  (anu  not 

a 

s.,  , , s , s.„  , ) need  be  considered  for  each  j . The 

jk  +1’  ok  +2*  * JK.-l' 

J J J 

index  k^  is  updated  appropriately  in  Step  3. 

In  Step  2,  only  those  j (1  < j < N)  are  considered  for  which 

b.  < b.,  since  any  b.  which  is  already  equal  to  its  upper  bound 
JO  0 

obviously  cannot  be  feasibly  increased  further.  As  discussed  above,  the 


variable  b which  is  chosen  to  be  increased  next  is  reset  to  its  next 
r 

larger  breakpoint  value  p . (Note  that  k is  updated  first  in 

r K r 

r 

Step  3>)  The  breakpoints  p used  in  Step  3 refer  of  course  to  the 

r 

r 

(all-integer)  breakpoints  of  the  modified  functions  z^(bj), 
j = 1,  2,  ...  , N. 

The  subsets  F which  are  to  be  searched  differ  from  one  another 
n 

in  the  values  of  the  bounds  b.,  b j = 1,  2,  . . . , N,  on  the  allo- 

1 J 

cation  variables.  After  the  initial  set  F^,  therefore,  we  may  not 

have  b.  = p.,  for  all  j = 1,  2,  ...  , N in  each  subset  F , and 
-j  jl  ^ n’ 

* 

hence  some  care  must  be  exercised  in  initializing  b.  and,  correspond- 

J 

ingly,  the  index  k . , j = 1,  2,  ...  , N,  in  Step  1.  Given  that  this 
J 

is  properly  accomplished,  however,  the  algorithm  above  can  be  used  to 
resolve  (MP)  at  each  node  n in  the  search  tree. 


4.3  Using  the  Master  Problem  in  the  RBBA 


The  bounding  step  (Step  3)  in  the  Rudimentary  Branch-and-Bound 
Algorithm  now  can  be  described  in  greater  detail.  Recall  that  the 
objective  in  Step  3 of  the  RBBA  is  to  determine  whether  the  current 
set  Fn  can  possibly  contain  any  allocations  which  lead  to  a better 
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overall  solution  to  (p)  than  the  incumbent,  and  if  so,  to  select  one 


such  allocation  for  further  examination.  This  objective  is  accomplished 


by  constructing  and  solving  the  Master  Problem  as  discussed  in  Sections 


Suppose  at  a given  node  n in  the  search  tree,  the  subset  F of 


feasible  allocation  is  to  be  considered  next.  Using  the  algorithm  of 


Section  4.2,  the  optimal  (integer)  solution  b.,  j = 1,  2 

J 

(MP)  is  easily  obtained.— ^ The  optimal  objective  value, 


provides  an  upper  bound  on  the  maximal  objective  value  in  problem  (4) 


If  U < z , where  z is  the  current  incumbent  objective  value,  the 


set  F is  fathomed.  If  U > z 


however,  the  allocation  b 


immediately  presents  itself  as  a natural  choice 


for  the  next  allocation  to  be  considered  in  Step  4 of  the  RBBA 


Consider  again  now  the  branching  procedure  presented  in  Chapter  3 


Suppose  that  when  subproblem  (S.  ) is  solved  in  Step  4 of  the  RBBA,  it 


is  discovered  that  the  allocation  b 


is  domi 


nated.  As  discussed  in  Chapter  3,  this  is  the  case  whenever  the  optimal 

(J0>*  , (d0)  (dJ* 

integer  solution  x in  (S.  ) uses  an  amount  b!  = A x < b 


of  the  linking  resource.  After  branching  on  b . in  Step  5 of  the  RBBA 


— This  solution  is  optimal  for  (MP)  in  the  sense  that  it  generates  the 
maximal  objective  value  which  (MP)  can  attain  over  the  set  F . Note, 

however,  that  the  allocation  b.  = b*,  j = 1,  2,  ...  , N,  still  may  not 

J J y. 

be  optimal  for  (P)  •,  that  is,  it  is  possible  that  v(P|(b1,  ...  , bN))< 

v(p|(b^,  ...  , b^))  for  some  other  allocation  (b^,  ...  , b^)  e Ffi, 

even  though  v(MP|(b*,  ...  , b*))  > v(MP|(b',  ...  , b ' ) ) . 


the  allocation  in  subproblem  (S.  ) is  "fixed"  at  the  level  b 


at  the  next  entry  to  Step  3*  Since  each  of  the  slopes  s 


and  break 


is  unchanged,  it  is  clear  that  in  resolving 


points  p 


algorithm  above  will  be  selected  in  the  same  order  (and  increased  by 
the  same  amounts  in  Step  3)  as  before,  with  the  exception  that  this 
time  the  allocation  to  (S.  ) cannot  be  raised  above  b.  = b'.  . Con' 

' -I  T 1 


sequently,  if  the  previously  optimal  allocation  was  saved,  the  Master 


Problem  need  not  be  resolved  from  scratch.  Instead,  starting  with  the 


(fixed),  (Mf>)  can  be  efficiently 


reoptimized  simply  by  using  the  algorithm  above  to  reallocate  the 


of  the  linking  resource  among  the  other 


’additional"  units  b 


subproblems  (S.) 


The  fact  that  the  allocations  to  certain  subproblems  are  fixed 


as  a result  of  the  branching  procedure  also  allows  the  upper  bound  U 


defined  in  (2C)  to  be  tightened.  Consider  an  arbitrary  node  n in 


Correspondingly,  let 


The  task  of  finding  an  optimal  solution  to  the  Master  Problem  at  node  n 


in  effect  reduces  to  that  of  optimally  allocating  the  remaining  amount 


of  the  linking  resource  among  the  unfixed  subproblems, 


represents  the  level  at  which  the  allocation  to  (S.) 


where 


is  fixed.  The  bound  (20)  generated  by  the  optimal  solution  b = b. 

0 J 

( j = 1,  2,  . . , N)  to  (MP)  at  node  n may  therefore  be  replaced  by 

the  tighter  bound. 


IT  = 2 v(S  |b*)  + £ z,(bp  , 

JJ  d d 


where,  v(S.|b.),  j e J , is  the  (known)  optimal  objective  value  in 
JO  -L 

* 

(S.)  under  the  allocation  b.  = b..  Since  U < U,  and  U can  be 
v J J J 

obtained  at  approximately  the  same  computational  expense  as  U,  it  is 
clearly  preferable  to  use  the  bound  U'  in  attempting  to  fathom 
in  Step  3 of  the  RBBA. 

Finally,  it  should  be  noted  that  Step  0 in  the  algorithm  for  solving 

the  Master  Problem,  included  for  the  sake  of  completeness  above,  need 

only  be  executed  once  in  the  course  of  the  RBBA.  Specifically,  prior 

to  the  start  of  the  RBBA  (as  described  in  Chapter  2),  the  subproblems 

(S.),  j = 1,  2,  ...  , N,  are  solved  as  linear  programs  parametrically 
J 

in  b . to  obtain  the  functions  z.(b,).  These  functions  are  then  modi- 
J 3 J 

fied  in  accordance  with  the  procedure  presented  in  Section  4.2.  By 


storing  the  slopes  s and  breakpoints  p of  the  revised  functions 

Jh  jk 

z!(b.),  it  will  not  be  necessary  to  recompute  these  values  each  time 
J J 

the  Master  Problem  is  solved. 


L 


CHAPTER  5 


COMPUTING  BOUNDS:  MULTIPLE  LINKING  CONSTRAINTS  CASE 

The  procedures  presented  in  Chapter  h,  while  very  efficient  for 
computing  the  required  objective  function  bounds  in  the  presence  of  a 
single  linking  constraint,  are  inappropriate  for  this  task  when  the 
block  angular  integer  program  (P)  has  more  than  one  linking  constraint. 
The  major  difficulty  which  arises  with  m > 1 linking  constraints  is 
that  the  parametric  linear  programming  problems  (4-. 5)  now  become  the 
multi -parametric  linear  programs  (MPLP), 


z.(b^)  = max 
J 


c(d)x(j) 


subject  to 


A(j)x(j)  < b(j) 

x^  > 0 , 


(1) 


,(j) 


j = 1,  2,  ...  , N,  where  bVt"  = (b,  ...  , b .)  is  now  an  m-vector 

lj  mj 

of  parameters.  Although  several  multi-parametric  linear  programming 
algorithms  have  recently  been  proposed,— ^ in  general,  the  MPLP  is  a much 
more  difficult  problem  than  the  standard  (single -parameter)  parametric 
linear  program.  Consequently,  direct  methods  for  solving  problems  such 
as  (1)  are  computationally  too  cumbersome  for  use  within  the  present 
context. 


- See  Gal  and  Nedoma  (1972),  or  Van  de  Panne  (1975).  Duffin's  (197^) 
modified  Fourier-Motzkin  elimination  procedure  can  also  be  used  for  this 
purpose. 
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A second  complication  which  arises  in  the  presence  of  multiple  linking 


constraints  is  in  the  structure  of  the  Master  Problem  itself: 


N 


maximize 


subject  to 


(j)' 


2 , 

j=l  J 


N 

B * 

j=i 


W < b 
( j ) < ^ ( j ) ^ 


(2) 


< b' 


integer,  j = 1,2, . . . ,N 


,(J)- 


While  it  is  again  easy  to  see  that  the  z.(bv<J  ) defined  in  (l)  are  concave 

and  piecewise  linear,  each  is  now  a function  of  several  variables. 

Therefore,  the  objective  function  in  problem  (2)  is  not  separable,  and 

the  solution  procedure  developed  in  Chapter  ^ can  no  longer  be  used. 

The  bounding  method  proposed  in  this  chapter  for  the  multiple  linking 

constraints  case  is  nonetheless  quite  similar  in  spirit  to  the  method 

discussed  in  Chapter  U.  As  before,  a Master  Problem  of  the  form  (2)  is 

introduced,  and  its  use  in  the  RBBA  is  entirely  analogous  to  the  use  of 

problem  (MP)  in  the  single  linking  constraint  case.  For  the  reasons 

mentioned  above,  however,  the  functions  z.(b^)  are  not  wholly  computed 

J 

in  advance;  rather,  relevant  "segments"  of  each  z.(b^^)  are  generated 

3 

in  the  course  of  the  algorithm.  Moreover,  because  of  the  loss  of  separ- 
ability, a different  linearization  procedure  for  the  concave  integer 
program  (2)  is  required. 


5 . 1 The  Bounding  Procedure 

In  order  to  compute  an  upper  bound  on  the  maximum  objective  value 
which  (P)  can  attain  over  a given  subset  of  feasible  allocations, 

we  utilize  the  dual  multipliers  associated  with  the  linking  constraints 
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in  the  LP-optimal  solution 


for  the  subproblems 


Theorem  1:  Let  z.  denote  the  optimal  LP  objective  value  in  the 


subproblem  of  (p),  given  the  allocation  b 


denote  the  LP-optimal  dual  multiplier 


associated  with  the  constraint 


for  any  other  allocation 


we  have 


Proof : Consider  the  LP  relaxation  of  the 


subproblem  under  the 


This  may  be  written  as 


maximize 


subject  to 


where 


The  dual  of  (5)  is 


minimize  y(«3)p(j)  ^ 


subject  to 


y(j)M(j)  > C(J) 

y^  > 0 . 


(6) 


Let  and  y^  denote  the  optimal  solution  to  (5)  and  (6)  respec- 

tively. £y  strong  duality, 


z.  = c^^^  = y^)p(0) 

J " 


(7) 


Now  suppose  that  the  right-hand  side  in  (5)  is  changed  to  p 


(j)‘ 


where 


+ •"*  > •••  > o)  , 


(8) 


and  where  Ab.  . , 1 < i < m,  is  given  by  (3).  The  dual  to  this  revised 
J 

problem  is 


minimize  y(<3)p(J)  ^ 


subject  to 


y(j)M(j)  > C(3) 
y^  > 0 . 


(9) 


Then,  since  y^  is  still  feasible  for  (9),  it  follows  by  weak  duality 
that  z'  < yvd^p^d'  , where  z\  is  the  optimal  objective  function 
value  in  the  revised  version  of  problem  (5).  Thus, 

”,  < y(j)p(j)’ 


= y(jVd)  + £ y[J)Ab  , from  (8) 


i=l 


m 


= z 


j + ^ ^j)Abu 
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, from  (7)  . 


is  the  optimal  solution  to  (6),  the  dual  of  (5),  the 


first  m components  of  y 


and  (4)  follows  by  noting  that  the  optimal  integer  solution  in  the  j 
subproblem  (corresponding  to  the  allocation  b'°'  ) must  yield  an  objec 


tive  value  le.  j than  or  equal  to  z 


) given  by  (4)  can  be  rewritten  as 


The  upper  bound  on  v 


follows 


),  from  (7)  and  (3) 


from  the  definition  of 


is  independent  of  the  allocation  . Again  noting 

. . for  1 < i < m,  we  have  from  (4)  and  the  last  equation 


so  that  z 


above  that 


v(S.  |b'^  ) < z_  + S,  ,b;  . 
J - oj  g.  ij  10 


Once  the  constant  z has  been  computed,  the  bound  (ll)  is  a 

'“'d 

function  only  of  the  allocation  b'J  Hence,  in  practice  it  will  be 
more  convenient  to  use  (ll)  than  (4).  Since  the  allocation  b^  in 
(11)  is  arbitrary,  by  summing  over  j = 1,  2,  ...  , N,  the  maximal 
objective  value  in  (p)  under  any  allocation  (b^,  . . , b^)  can  be 


bounded  by- 


vf?l(b(1),  ...  ,b»»<  S + £ ) 


In  general,  different  allocations  blJ'  lead  to  different  LP-optimal 

bases  in  (S.),  and,  consequently,  to  different  sets  of  optimal  dual 
d 

multipliers  [\  , 1 < i < m}.  Each  of  these  sets  can  be  used  to  generate 

-**d 

an  upper  bound  on  v(S.|b^)  as  in  (ll).  Indexing  the  different  sets 

J 

by  the  superscript  k,  and  letting  L ^ denote  the  total  number  of  such 

sets  associated  with  the  subproblem  (S.),  we  have 

J 

v(S  |b(J))  < min  (z<*}  + £ \(*}b  ) , (15) 

d < \r  < T -i  1d  J-d 


1 < k < L. 

- I 


where  z^.  is  computed  from  (10),  for  k = 1,  2,  ...  , L . • Hence, 

defining  z.(b^)  by 
d 

zi(bl,jl)r.  min  + C'  3 = 1,2,-  ,N  , (14) 


1 < k < L. 

- - 3 


ij  1 3 


- We  note  that,  by  the  way  in  which  (12)  was  obtained,  the  sets  of  dual 
multipliers  (\. .,  1 < i < m)  need  not  be  related  to  each  other  for 

f i)* 

different  j = 1,2,  . ,N.  In  particular,  the  allocations  bVlJ/  used  to 

obtain  l\.  , 1 < i < m),  j = 3,2,..  ,N,  (as  in  Theorem  l)  are  not 
J N 

required  to  satisfy  J)  b ^ < b. 

j=l 
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leads  to  the  Master  Problem  (2). 


To  convert  the  concave  (non-separable)  problem  (2)  into  a standard 
mixed-integer  linear  programming  format,  the  objective  function  components 
z .j(b^)  are  replaced  by  the  auxiliary  variables  z.,  j = 1,  2,  ...  , N. 
The  z are  not  restricted  in  sign,  but  from  (14),  are  required  to  satisfy 


z.  < zi^  + 2/  x!k)b..  for  each  k = 
3 ~ O3  if/1  i3  13 

1,2,  ...  , L . : 

J 

N 

(MP)  maximize  £ z , 

j=l  J 

(15) 

subject  to 

N 

£ £ bi 

j=l 

i=l,2 , , . . ,m 

(16) 

z . y x(k)b  < 2(k) 

3 kij  bij  - Oj  * 

k=l , 2 , • • • ,Lj , 

(17) 

bij  £ bi;i  £ V bij  integer’ 

1=1 ,2 , ' * • 

(18) 

Since  the  objective  (15)  in  (MP)  is  to  maximize  the  sum  of  the  auxiliary 

variables  z.,  it  is  clear  that  in  the  optimal  solution  (zT,  b^  ), 

J 3 

j = 1,  2,  ...  , N,  the  implied  equality  in  (14)  will  be  naturally  satis- 
fied^  i.e.,  we  will  have 


z = min  (z 

J 1 £ k 1 Lj  J 


w + * Jk> 


Si) 


O kij  bij)  =■  MbU  )>  = 1,2,. ..,N  , 


i=l 


as  desired.  Thus,  the  Master  Problem  (MP)  above  is  equivalent  to  problem 

(2)  when  the  concave  functions  z.(b^)  are  as  given  by  (14). 

J 

Each  of  the  slopes  s.,  of  the  different  segments  of  z (b.)  defined 

3*  J 3 

in  Chapter  4 is  exactly  the  dual  multiplier  associated  with  the  (single) 

linking  constraint  in  the  basis  that  is  optimal  for  (S.)  at  b.  = p... 

J 3 Jk 
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Thus,  there  is  a strong  analogy  between  the  Master  Problem  (MP)  above 

and  that  introduced  in  Chapter  4 for  the  single  linking  constraint  case. 

(k) 

Each  of  the  sets  of  multipliers  {\.  . , 1 < .i  < m ] , k = 1,  2,  ...  , L., 

ij  - - J 

defines  a different  segment  (hyperplane)  of  the  concave  and  piecewise 

wv.  ry  t U ' ^ Tv.  4V  w 4-  4-U  w 4- « w )v  « -P  4 « -!  4*  1 . . ,4-4v.._ 


linear  function  z.(b 
J 


In  fact,  the  task  of  explicitly  computing 


the  z.(bu  ) in  (lit-)  is  tantamount  to  solving  the  multiparametric 
J 

linear  program  (l).  As  has  been  previously  noted,  however,  this  can 

be  a computationally  unattractive  approach  for  our  purposes 

Rather  than  solve  the  MPLP  (l)  directly,  therefore,  we  note  that 

'k)  , (k) 

the  necessary  multipliers  A.) ^ ' (and  constants  z^.  ) can  easily  be 

obtained  as  a by-product  of  the  subproblem  solutions  computed  in  the 

course  of  the  RBBA  Specifically,  suppose  that  at  some  point  in  the 

algorithm,  i.  distinct  sets  of  multipliers  . , 1 < i < m]  (k  = 

J J 

1,  2,  ..  , i .)  have  been  obtained  in  the  j subproblem,  where 

J 

1 < l- . < L.  A '’relaxed"  master  problem  of  the  form  (MP)  can  be  con- 
J J 

strutted  in  which  some  of  the  constraints  (17)  are  omitted.  The  Relaxed 
Master  Problem  is 


(RMP)  maximize  7}  z . , 
j=i  J 

subject  to 


b . . < b. 
iJ  - i 


, i=l,2, ...,m 


- (k'i  fk' 

zj  ‘ xij  bij  - zoj'’  k=1’2>  •■>£y  J=i»2,...,n  (2i) 


b..  < b..  < b.  .,  b..  integer,  i=l,2, . . . ,m,  j=l,2, . . . ,N  . (22) 

-i,1  - ij  - iy  ij  ’ v 


In  considering  the  set  F of  feasible  allocations,  the  optimal 
* ^ 

solution  (z.,  b^  ),  j = 1,  2,  ...  , N,  to  (RMP)  provides  a valid 
J 

N * 

upper  bound  U = 2/  z • on  bhe  maximal  objective  value  which  can  be 

j=l  J 

attained  in  (P)  under  any  allocation  in  F^.  If  F^  is  not  fathomed 

(l)*  (n)* 

by  U,  the  allocation  (bv  , ...  , b^  ) may  be  used  in  solving  the 
subproblems  in  Step  4 of  the  RBBA.  If  any  linear  programming  based  solu- 
tion procedure  is  used  (e.g.,  a branch-and-bound  method  such  as  was 
outlined  in  Section  1.2),  it  is  well-known  that  the  dual  multipliers 

1 < i < m,  appear  in  the  optimal  LP  solution  to  (S.)  as  the  reduced 
r J 1 

cost  coefficients  associated  with  the  linking  constraint  slack  variables. 

(k) 

Thus,  while  it  may  be  advantageous  to  derive  a few  sets  . , 1 < i < m) 

of  multipliers  initially  in  each  subproblem,  remaining  sets  can  be  obtained 

at  little  additional  computational  expense  as  the  search  proceeds.  In 

this  way,  much  of  the  work  involved  in  explicitly  computing  the  functions 

z.(b^)  defined  in  (14)  can  be  avoided. 

J 

(k) 

One  procedure  by  which  suitable  initial  sets  of  multipliers  f\: . , 

* J 

1 < i < m}  may  be  determined  is  begun  by  specifying  an  arbitrary  allo- 

/ . j*  / . )* 

cation  , j = 1,  2,  . . . , N,  in  each  subproblem.  These  b^J 

should  be  chosen  to  ensure  that  each  of  the  subproblems  (S.)  has  an  LP- 

J 

optimal  solution  which  is  finite,  but  it  is  not  necessary  that  the  overall 

-X-  * 

allocation  (b^  , ...  , b^  ) itself  be  feasible;  that  is,  it  is  not 
necessary  that  the  constraint 


N , .x* 

t b(j)  < b 

j=l 


be  satisfied.  When  each  of  the  subproblems  is  solved  as  a linear  program, 
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the  resulting  sets  of  dual-optimal  multipliers  can  be  used  to  construct 
a first  relaxed  master  problem,  say  (RMP^).  This  problem  may  in  turn 
be  solved  to  generate  a second  (feasible)  allocation,  thereby  establishing 
an  iterative  routine.  In  particular,  on  the  iteration  (k  = 1,2,...), 

the  sub problems  are  resolved  as  linear  programs,  using  the  optimal  allo- 
cation from  (RMP.  ).  Any  new  sets  of  multipliers  discovered  in  the  process 
are  used  to  augment  (RMP  ) to  obtain  the  next  relaxed  master  problem 
(RMP,^+^)-  The  procedure  terminates  at  iteration  K when  no  new  sets  of 
multipliers  result  from  the  optimal  allocation  in  (RMP„) . The  final 

problem  (RMP-  ) then  serves  as  the  initial  relaxed  master  problem  in 
K 

the  RBBA. 

In  many  practical  situations,  a good  starting  allocation 

, * * N * 

(bv  , ...  , b^  ) may  be  readily  available  from  previous  experience 
with  the  problem.  If  no  such  allocation  can  be  easily  determined,  however, 
one  may  begin  by  specifying  b ^ ' = b^\  j = 1,  2,  ...  , N,  where 
b^  is  the  initial  vector  of  upper  bounds  on  the  linking  resource  allo- 
cations in  the  subproblem.  (Methods  for  computing  b^  have 

been  presented  in  Section  2.),)  If  any  subproblem  (S.)  is  then  found 

t) 

to  be  infeasible,  it  is  clear  that  the  block  angular  integer  program 
(P)  must  itself  be  infeasible. 

Note  that  z.,  the  optimal  LP  objective  value  in  (S.)  under  the 
allocation  b J , must  satisfy 

z < z*  = min  (.£>  + 2 ^fhj)  • (25) 

J J 1 < k < i . U:I  i=l  lo 

- ~ 3 

* / . v * 

If  equality  holds  in  (23),  the  bound  z.  on  v(S.|b^  ) obtained 

J 3 

from  (RMP)  is  as  tight  as  that  which  would  have  been  produced  by  solving 
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the  full  Master  Problem  (MP) . If  z.  < z.,  however,  it  follows  by 

J J 

strong  duality  that  (adopting  the  notation  of  Theorem  1 again) 


7(d)p(j)  < min  (z<*>  + £ ^k)b?.)  . 

. v Oi  L-/  ni  ii 


1 < k < i . 

- - 3 


ij 


The  inequality  (24)  in  turn  implies  that  the  set  of  dual  multipliers 

(V1) 

[\ . , 1 < i < m)  obtained  in  solving  (S.)  is  distinct  from  any  of 

J 

( k) 

the  previous  sets  (\: . , 1 < i < m),  k = 1,  2,  ...  , i ..  Moreover, 

i J 1 


. 1 


(^+1)  m (i  +1)  ^ 

Z . = y(j)p(t]  ' = z 3 + f.  . 3 b.  . , 

■>  Oj  1J  n ’ 


1J  11 


this  new  set  of  multipliers  gives  rise  to  precisely  that  (binding)  con- 
straint of  (MP)  which  was  "missing"  from  (RMP).  Thus,  the  new  multipliers 
generated  in  the  course  of  the  RBBA  are  in  some  sense  exactly  (and  only) 
those  which  are  most  relevant.  By  the  same  argument,  it  also  seems 
reasonable  to  expect  that  the  iterative  procedure  described  above  will 
indeed  lead  to  suitable  initial  sets  of  multipliers  for  (RMP). 

A second  advantage  in  utilizing  the  relaxed  master  problem  (RMP) 
rather  than  constructing  the  full  master  problem  (MP)  is  that,  in  general, 
(RMP)  has  fewer  constraints  than  does  (MP).  Practical  experience  has 
indicated  that  the  computational  effort  required  to  solve  linear  pro- 
gramming problems  increases  approximately  as  the  cube  of  the  number  of 
functional  constraints.  Since  most  integer  programming  algorithms  are 
based  on  linear  programming  techniques,  it  is  reasonable  to  expect  that 
(RMP)  will  require  less  solution  time  than  would  (MP).  Of  course,  this 
savings  is  obtained  at  the  cost  of  possibly  reduced  fathoming  capabilities 
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in  the  RBBA,  since  the  objective  function  bound  resulting  from  (RMP) 
may  not  b<  as  tight  as  that  provided  by  (MP).  However,  since  most 


fathoming  will  tend  to  occur  at  later  stages  in  the  search,  and  since 

(RMP)  may  be  augmented  with  the  relevant  additional  constraints  (17) 

(k) 

as  new  sets  of  multipliers  . , 1 < i < m)  are  discovered  in  the 

ij  - - 

course  of  the  search,  the  loss  may  well  be  minimal. 

The  relaxed  master  problem  (RMP)  may  be  solved  by  any  standard 

mixed-integer  programming  algorithm.  Because  the  auxiliary  variables 

z.  are  not  sign-restricted,  however,  the  constraints  (21)  serve  merely 
J 

to  supply  upper  bounds  on  the  values  which  these  variables  may  assume. 
In  this  sense,  the  only  functional  constraints  which  appear  in  (RMP) 
are  the  linking  constraints  (20) . Consequently,  integer-feasible  solu- 
tions to  (RMP)  are  easily  obtained,  and  this  fact  can  be  exploited  to 
improve  the  overall  efficiency  of  solution. 


In  particular,  consider  the  situation  when  an  enumerative  algorithm 

is  used  to  solve  (RMP) . The  integer  restrictions  on  the  allocation 

variables  b. . are  relaxed,  and  (RMP)  is  first  solved  as  a linear  pro- 
i-J 

ry,  *1  ) N 

gram.  Let  (z.,  bvu/),  j = 1,  2,  ...  , N,  denote  the  optimal  LP  solution. 
J 

If  each  b . . , 1 < i < m,  1 < j < N,  is  integer-valued,  (z.,  bVd  ) is 

J 

optimal  for  (RMP),  and  we  are  done.  Otherwise,  however,  let  f. . = 

b..  - [ b . . ] , 1 < i < m,  1 < j < N,  denote  the  fractional  part  of  b. 
ij  ij  - - - - ij 

and  define 


I.  = f.u  i - 1>  ->  •••  > m 
j=l  J 


Then,  for  i = 1,  2,  ...  , m. 
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"mu  mmiMi 1 1.1  j"'i 


1 


T)  tb. .1  + V f. . 

A 1J  A « 


and  it  follows  that  1^  must  be  an  integer.  Moreover,  by  (25),  since 

0 < f. . < 1,  j = 1,  2,  ...  , N,  it  is  clear  that  0 < I.  < N, 

1 J 1 

1 = 1,  2,  ...  , m.  For  each  i,  then,  I.  of  the  b.  . (j  = 1,2,...,N) 

i l j 

may  be  arbitrarily  selected  to  be  rounded  up  to  [b.  .]  + 1,  while  the 

3-J 

others  are  rounded  down  to  [b.  .J.  One  reasonable  criterion  by  which 

J 

to  decide  which  I.  of  the  b.  . to  round  up  is  to  choose  those  b.  . 

i ij  ij 

with  the  largest  fractional  parts  f _ ; 

C ~ 

[b..]  + 1,  if  f . . is  among  the  I.  greatest  members 

b!  . = J of  (f.  .,  1 < j < N}  (26) 

ij  ij  - - v 

[b . . 1 , otherwise  , 

v lj 


for  i = 1,  2,  ...  , m.  Since 


N N N 

S 5{,  - 2 1‘iJ  * h - S " »i  U - 1.  2>  •••  - -)  - 


A A 1J 


tne  rounded  solution  b. . = b* . (1  < i < m,  1 < j < N)  satisfies  the 

lj  10  “ ~ - - 

constraints  (20)  of  (RMP). 


The  maximal  values  z!  which  each  auxiliary  variable  z.  may  then 

J J 

a sume  can  be  readily  computed  from  the  constraints  (21).  The  integer- 

~ j ) 1 \ 

feasible  solution  (z!,  u ),  j = 1,  2,  ...  , N,  immediately  provides 

11  N 

. itial  incumbent  solution  for  (RMP)  (with  objective  value  z!), 

j = l J 


which  may  then  be  used  to  enhance  the  power  of  subsequent  fathoming  tests. 
Non-integer  solutions  produced  at  other  nodes  in  the  course  of  solving 
(RMP)  may  be  similarly  rounded  in  the  hope  of  quickly  locating  improved 
integer  feasible  solutions. 

In  limited  preliminary  computational  tests,  this  simple  rounding 

procedure  was  found  to  be  surprisingly  effective.  For  block  angular 

problems  with  all-integer  cost  coefficients  c^  ( j = 1,  2,  . . . , N)  - 

in  which  case  the  auxiliary  variables  z.  may  be  integer-restricted  as 

J 

well  in  (RMP)  - the  rounded  LP-optimal  solution  b! . was  usually  optimal. 

— J 

More  importantly,  with  only  rare  exceptions  it  was  found  that  whenever  the 
* N * 

bound  U = 2/  z a was  sufficient  for  fathoming  a given  subset  F of 

i=l  ^ n 

J N 

feasible  allocations,  the  bound  U = 2/  z • was  also  sufficient  for 

0=1  J 

this  purpose.  Consequently,  little  fathoming  power  is  lost  by  terminating 

the  solution  procedure  for  (RMP)  as  soon  as  the  optimal  LP  solution  has 

been  obtained.  If  the  bound  U does  not  fathom  F , the  (integer-feasible) 

n ' ° 

allocation  bVJ  = (b_,  ...  , b^),  j = 1,  2,  ...  , N,  given  by  (26) 
may  be  selected  next  for  examination  in  Step  4 of  the  RBBA.  The  compu- 
tational effort  required  to  solve  the  mixed-integer  programming  problem 
(RMP)  is  thereby  reduced  to  little  more  than  solving  its  linear  program- 
ming relaxation. 

Except  for  the  differences  in  solution  procedures,  the  relaxed  master 
problem  (RMP)  is  used  in  the  RBBA  in  exactly  the  same  way  as  is  the  corres- 
ponding master  problem  for  the  single  linking  constraint  case.  Remarks 
analogous  to  those  in  Section  4.3  applj  here  as  well.  In  particular, 
we  note  that  the  objective  function  bound  obtained  by  solving  (RMP)  can 


be  tightened  by  using  the  known  integer  programming  objective  values 
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corresponding  to  any  fixed  subproblem  allocations  b^  . As  before, 

in  considering  a given  subset  of  allocations  F , let 

n 

J,  = (j|The  allocation  to  subproblem  (S.)  is  fixed  in  F ) . 

1 0 n 

J2  = {j|l  < j < N}  - Jx  . 

By  appropriately  adjusting  the  right-hand  sides  in  the  constraints  (20) 

of  (RMP) , the  variables  (z  , b^),  j e J, , may  be  dropped  from  the 

J •*- 

problem,  together  with  the  corresponding  constraints  (21).  The  resulting 
(reduced)  problem, 


< 


maximize  £ z . , 

jeJ2  J 

subject  to 


z . 
J 


2 

jeJ2 


b. . < b. 
xj  - i 


xj' 


m 


2 

i=l 


. < z 
13  13  - 


(k) 

Oj 


i = 1,  2, 


k = 1,  2, 


b. . < b. . 
-13  - 13 


integer 


i = 1,  2, 


(27) 

) j e J2 
> ni,  j e , 


is  computationally  less  expensive  to  solve  than  the  full  problem  (RMP) 

itself.  The  maximal  objective  value  which  (p)  can  attain  over  the  set 

F can  then  be  bounded  by 
n 

U = £ v(s  |b(j)  ) + £ z*  . 

3 eJ1  J j eJ2  0 

Alternatively,  if  the  solution  of  problem  (27)  is  terminated  after  the 
LP-optimal  solution  (z^,  3 € J2>  is  obtained  (as  suggested 

above),  we  have 
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u = £ v(s ,!b(o-}  ) + » 

A ~ T w ^ T 


5 . 2 The  Decomposition  Algorithm 

We  are  now  in  a position  to  present  a detailed  statement  of  the 


various  steps  of  the  decomposition  algorithm..  Although  the  three  major 
components  of  the  algorithm  (branching,  bounding,  subproblem  solution) 
have  already  been  discussed  to  some  extent,  the  following  exposition 
will  serve  to  indicate  more  precisely  how  these  functions  are  coordi- 
nated. In  addition,  we  include  in  the  algorithmic  description  below 
several  refinements  not  previously  mentioned.  These  steps  are  in  a 
sense  optional,  but  in  practice  can  significantly  improve  the  overall 
efficiency  of  the  algorithm,  A brief  discussion  of  these  and  other 
points  follows  the  statement  of  the  algorithm. 

For  both  the  single  and  multiple  linking  constraint  cases,  the 
algorithm  for  decomposing  block  anguiar  integer  programs  is  divided  into 
two  phases,  Tr,  Phase  1,  the  initial  computations  required  to  construct 
the  appropriate  master  problem  are  performed.  The  search  for  an  optimal 
allocation  of  the  linking  resources  to  the  subproblems  is  then  carried 
out  in  Phase  2.  Flow  diagrams  for  the  two  phases  of  the  algorithm  are 
presented  in  Figures  1 and  2,  respectively.  In  these  diagrams  the 
notation  SLC  ana  MAC  is  used  to  indicate  branches  which  apply  to  the 
single  and  multiple  linking  constraints  cases,  respectively. 


Decomposition  Algorithm  for  the  Block  Angular  Integer  Program  (p) 


Phase  1:  Construction  of  Master  Problem 


Step  0.  Input  problem  data.  Compute  upper  and  lower  bounds  b 


on  the  allocation  variables  b 


subproblem,  j = 1,2 


If  (P)  has  only  a single  linking  constraint,  go  to  Step  1.  Otherwise 


Step  1.  (Single  linking  constraint.)  For  j = 1,  2,  . . . , N,  solve 

subproblem  (S.)  as  a linear  program  parametrically  in  the  amount  b. 

J J 

of  the  linking  resource  allocated  to  (S.),  b.  < b.  < b..  Modify  the 

J «3  J J 

resulting  parametric  functions  z . (b  . ) to  eliminate  non-integer  break' 


points.  Go  to  Phase  2,  Step  0 


Step  2.  (Multiple  linking  constraints.)  Set  £ . = 0,  j = 1,  2 


known  for  which  each  subproblem  has  a finite  LP-optimal  solution,  set 


Otherwise,  set  b 


Step  3.  If  j > N,  go  to  Step  6.  Otherwise,  solve  subproblem  (S.) 


as  a linear  program,  using  the  allocation  b 


in  the  linking 


be  the  (m-m. ) -vector  of  optimal  dual  multipliers 


, as  obtained  in  Step  3*  If  no  finite  optimal  y 


then  reset 


Otherwise 


rn+i  1 


Then  reset  n = n + 


programming  relaxation  of  (RMP) , using 


ponding  constants  z 


construct  the  constraints  (21).  For  any  1 < j < N such  that  l.  = 0 


to  the  LP-optimai  solution  to 


Phase  2:  Search  for  Optimal  Alloca ti on 


tep  0.  If  an  integer  solution  to  (P)  is  known  a priori,  set  z equa 


to  its  objective  value  and  store  the  solution  as  the  current  incumbent 


Otherwise,  set  z 


Step  1.  If  the  list  of  stored  subsets  F,  is  empty,  stop;  the  current 


incumbent  solution  is  optimal  (if  none  has  been  found,  (P)  is  infeasible.) 


Otherwise,  select  the  last  subset  F on  the  list  as  the  next  one  to  be 


examined.  Update  the  allocation  variable  bounds  b 


appropriately 


to  correspond  to  those  defining  the  subset  of  feasible  allocations  F 


Correspondingly,  update  the  sets  J.. , J ; specifically,  if  the  allocation 


reset 


to  subproblem  (S.)  was  fixed  previously,  but  is  not  fixed  in  F , reset 
J ^ 

Jx  = J1  - (j),  J2  = J2'J  (j).  Go  to  Step  2. 

Step  2.  (a)  Single  Linking  Constraint.  Solve  the  concave-separable 

master  problem  (MP)  (see  Sections  4.1,  4.2),  using  the  current  bounds 

-X-  * 

b.,  b..  Set  (b  , ...  , b ) equal  to  the  optimal  integer  solution.  Set 
J 0 i\i 


u . £ V(s.lb*>  * 1 *X>  , 


and  go  to  Step  3> 

(b)  Multiple  Linking  Constraints.  Using  the  multipliers 

(k)  (k) 

. , 1 < i < m)  and  corresponding  constants  z^.  (k=l,  2,  ...  ,i., 

j e Jg),  construct  the  relaxed  master  problem  (27).  For  any  j e 

for  which  i . = 0,  use  \{  P = 0,  1 < i < m,  in  (27).  For 

J Oj  ij  - 

f "j  J 

j € Jp,  set  ( z bvu  ) equal  to  the  optimal  LP  solution,  and  set 


u = Z v(S . |b  ^ ) + £ 


where  bVJ;  , j e J^,  is  the  current  level  at  which  the  allocation  to 

(S.)  is  fixed.  Go  to  Step  3* 

J 

/ \ * 

Step  3-  (a)  Single  Linking  Constraint.  If  U < z , go  to  Step  1. 


Otherwise,  go  to  Step  4. 

/ i * 

(b)  Multiple  linking  Constraints.  If  U < z , go  to  Step  1. 
Otherwise,  round  the  allocations  b^,  j e J , as  in  (26),  and  set  b^ 
j e J2,  equal  to  t?ie  resulting  integer- feasible  values.  Go  to  Step  4. 

Step  4.  Compare  the  current  allocations  b with  the  stored  list 

of  previously  examined  allocations  b^d'  (if  any)  in  subproblem  (S.), 

(V  (V  (v*  (jo),J 

j £ J2.  If  some  j e yields  b - s < b < b 
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cy  (j0)' 

(where  b - s represents  the  level  of  linking  resources  used 


(jj: 


in  the  optimal  solution  to  (S.  ) under  the  allocation  b 0 ),  then 


(J0)*  (J0)*  (j0) ,0  (J0)‘ 

set  s = b - (b  - s ) and  go  to  Step  10.  Otherwise, 


go  to  Step  5- 


Step  5.  Select  a subproblem  (S.  ),  j e J_,  by  either— 

J0  ° 


1/ 


(i)  j = min(j | j e J2) ; or 


(ii)  j = arg  min  TH,  where  T..  is  the  average  solution  time  for 


j e Jr 


(S.)  observed  in  the  previous  solutions  of 
J 


of  (Sj)  in  Phase  2;  or 


(iii)  j = arg  max  G.,  where  G.  is  the  average  gap  between  the  LP 


j el 


J * 

,(J> 


bound  z . (b'  ) obtained  in  Step  2 and  the 

J 


actual  corresponding  integer-programming  value 


/ . \ 1 

v( S | b ^ ) for  (S.)  observed  in  the  previous 

J J 


solutions  of  (S.)  in  Phase  2;  or 
J 


(iv)  = arg  max  X.,  where  X.  is  the  average  sum-of-slacks 


j C Jr 


J 

m , .v* 

V A 3) 


£J 


i=l 


observed  in  the  linking  constraints 


in  previous  solutions  of  (s.)  in  Phase  2. 

J 


Then  go  to  Step  6. 


17 


- The  notation  j = arg  min  T (arg  max  G.,  arg  max  X.)  is  used  to 


j e J2  0 j e J2  J 


j e Jr 


indicate  that  the  index  j.  is  to  be  chosen  so  that  T.  = min  T. 

0 J0  jSj2  J 


(G.  = max  G.,  X.  = max  X.). 

^ ir  t 3 3n  ■ , J 


J0  jeJ, 


'0  jeJ, 


! 
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Step  6.  Compare  the  current  allocation  b 


with  the  stored  list 


of  previously  determined  levels  of  optimal  linking  resource  usage  b 


max 


Step  7.  Solve  (S„.  ) using  the  allocation  b . If  a branch-and-bound 
J0 

method  is  used,  the  value  of  z computed  in  Step  6 provides  an  initial 


lower  bound  on  the  optimal  objective  value.  Then 


If  there  exists  a finite  optimal  solution  x 


If  m = 1 (single  linking  constraint  case) 


is  feasible,  but  the  optimal  solution  is  unbounded 


be  an  endpoint  of  the  ray  generating  the  unbounded 


solution,  and  set  s 


cussion  below.)  Set  z 


Step  8.  (Used  in  multiple  linking  constraint  case  only.)  Let  y 0 
be  the  (m+m.  )-vector  of  LP-optimal  dual  multipliers  for  (S.  ) under 


(as  determined  in  Step  7).  If  for  some 


the  allocation  b 


k - 1,  2,  ...  , Z . , 

<J 


ip  jj  ) (j0) 
7)  y ? P.  0 

i=l 


(k)  -^0^  (k) 

°J0  ^ U0 


1 < i < m , 


go  to  Step  9-  Otherwise,  reset  Z.  = Z.  +1,  and  set 

0 J 


^jj  io  JiJ  (dJ 


_(dA) 


J0  i=l 


S ym-i  h , >"<»  v, ' - y, 0 . i < i < - • 


Then  go  to  Step  9. 


Step  9.  Add  b 


to  the  list  of  examined  allocations.  Correspond- 


ingly, add  b 


(jo)  (jo) 


* - NWA' 

and  z.  = v(S.|b  ) to  the  lists  of 


optimal  linking  resource  usage  levels  and  associated  objective  values, 
respectively.  Go  to  Step  10. 


Step  10.  If  |J  I < 1,  go  to  Step  11.  Otherwise,  branch  on  the  alio- 

TV 

cation  variables  b as  described  in  Chapter  3.  (See  especially 

Figure  2 of  Chapter  3.)  Add  each  of  the  newly  formed  subsets  F,  to 

K 

-X-  -X- 

the  list  of  such  subsets.  If  b..  > b. . - s. . for  some  1 < i < m, 

1J0  1Jo  1Jo 

(jn)  (dJ  (jn) 

go  to  Step  1.  Otherwise,  revise  the  bounds  on  b to  b = b 

(j0)*  Jj0)  (j0)*  1/ 

- s » b = b ,-  and  reset  = J±  ij  (jQ},  Jg  = J2  - C jQ} - 
(The  allocation  in  subproblem  (S.  ) is  then  temporarily  fixed  at  the 

(j0)  (j0)*  (jQ)*  °° 

level  b = b - s . ) Go  to  Step  2. 


jj  , . (j0)’  (j0)' 

- If  in  Step  4 of  Phase  2 it  was  discovered  that  b - s < 


(jn)  (jn) 


(j0)' 

for  some  previously  examined  allocation  b , it 


— ( j 0 ) Jd0)  (jo)'  (jp) 

will  be  more  efficient  to  reset  b = min(b  ,b  ) > b u 
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MHW 


N 

•X-  -)f  -X-  -X- 

Step  11.  If  z . < z , go  to  Step  1.  Otherwise,  reset  z = J)  z., 

0=1  J 0=1  J 
-X-  -X- 

and  install  (x^  , ...  , ) as  the  new  incumbent  solution;  then 


go  to  Step  1. 


The  (optional)  Steps  4,  6,  9 in  Phase  2 of  the  decomposition  algorithm 


can  improve  algorithmic  efficiency  in  the  following  ways.  In  Step  4,  if 


(oQ)  ( . (‘V 


< b , the  solution  which  was  optimal  for 


(o0)  (J0)' 

(S.  ) under  the  allocation  b = b must  also  be  optimal  for  the 


0 , . X , . N* 

(Oq)  (j0) 

current  allocation  b = b .In  this  case,  therefore,  the  work 


to  resolve  (S.  ) can  be  avoided,  and  instead  we  can  jump  immediately  to 
°0 


the  branching  step  (Step  10).  Branching  on  the  allocation  variables 

(J0) 

b is  done  in  exactly  the  same  way  as  if  (S.  ) had  been  resolved 

(jJ  (jn)’  J0 


explicitly  under  the  allocation  b =b 


At  Step  6 in  Phase  2,  when  the  subproblem  (S.  ) must  be  resolved 


explicitly,  the  validity  of  the  lower  bound  z_  on  v(S.  |b  ) follows 

, u0)  ’°  (j0) 

from  the  observation  that  v(S . |b  ) must  be  nondecreasing  in  b 

J0 


If  a branch-and-bound  search  procedure  is  used  to  solve  (S.  ) in  Step  7> 


the  lower  bound  z is  immediately  available  for  use  in  fathoming  tests, 


and  can  thereby  reduce  the  total  computational  effort  of  solution.  How- 


ever, in  general,  any  subproblem  solution  method  may  be  used  in  Step  7 


in  the  single  linking  constraint  case,  and  any  method  yielding  the  LP- 

— (Jq) 

optimal  dual  multipliers  y as  a by-product  of  the  solution  may  be 


used  in  the  multiple  linking  constraint  case.  If  the  method  chosen  for 


subproblem  solution  is  not  of  the  branch-and-bound  type,  it  may  be  more 


efficient  to  omit  Step  6 in  Phase  2. 


I 

I 
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Input  data . Compute 
init.  bounds 


Solve  (S.)  as  LP  par 

ametriealty  in  b.(b. 
< b.  < b.).  J J 


Modify  z . (b  . ) to 

eliminate  noninteger 
breakpoints . 


'New  finite, 
optimal  dual  mul 
s.  tipliers? 


Figure  1:  Flow  diagram  for  Phase  1 of  the  decomposition  algorithm 
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J 
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b(j)  _ 
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t . 

^ Stop 


Figure  2a:  Flow  diagram  for  Phase  2 of  the  decomposition  algorithm  (continued  in  Figure 


Phase  2 of  the  decomposition  algorithm  (continued  from  Figure  2a) 
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Of  course,  the  computational  savings  resulting  from  Steps  4 and  6 
are  obtained  at  the  expense  of  increased  storage  requirements.  The 


necessary  data  storage  is  performed  in  Step  9 of  the  algorithm.  Some 
savings  of  storage  space  is  possible  by  noting  that  if  for  some  previous 


allocation  b 


(V 


^0^  JV  (J0} 

b - s = b - s and  b > b 


(J0)*  (j0)*  (d0)*  O0)’ 

then  the  new  data  b , b - s may  replace  b , 


(3n)V 


in  storage.  (Since  v(S.  |b  0 ) = v(S.  |b  0 ) 


I v d0 

in  this  case,  v(s.  |b  ) need  not  be  rewritten.) 

J0 

Four  different  heuristic  procedures  for  subproblem  selection  are 
suggested  in  Step  5 of  Phase  2.  Of  these,  the  first  has  the  advantage 
that  it  is  clearly  least  expensive  computationally.  Rule  (ii),  however, 
can  be  more  effective  in  practice,  especially  if  some  subproblems  require 
significantly  longer  solution  times  than  others  (e.g.,  if  some  subproblems 
are  much  larger  than  others).  By  solving  the  easier  subproblems  first 
(at  which  point  it  becomes  possible  to  use  the  exact  integer  programming 
objective  values  for  these  subproblems  in  computing  the  bound  U in 
Step  2),  fathoming  will  often  occur  before  it  becomes  necessary  to  solve 
the  more  difficult  subproblems. 

On  the  other  hand,  the  easiest  subproblems  to  solve  do  not  always 


provide  the  most  information  for  subsequent  use  in  the  decomposition 
algorithm.  The  motivation  for  rule  (iii)  is  that  selecting  the  subproblem 

(d0)  , 

(S.  ) (jn  e J ) for  which  the  average  gap,  z (b  ) - v(S.  |b  ), 

U d J0  J0 
is  greatest  may  lead  to  the  best  improvement  in  the  tightness  of  the  bound 
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U computed  in  Step  2 at  the  next  iteration.  The  hope  is,  of  course, 

that  by  enhancing  the  power  of  the  fathoming  test  in  Step  3>  the  total 

number  of  subproblem  solutions  required  in  the  course  of  the  search  will 

be  reduced.  Alternatively,  the  use  of  rule  (iv)  in  Step  5 can  lead  to 

the  greatest  inprovement  in  the  allocations  generated  in  Step  3 of  the 

algorithm;  following  the  solution  of  (S.  ) in  Step  7 and  subsequent 

J0 

(V* 

branching  in  Step  10,  the  slacks  s are  reallocated  to  the  remaining 

unfixed  subproblems  in  Step  3-  Hence,  it  also  would  seem  reasonable  to 
solve  first  those  subproblems  (S.)  which  consistently  give  rise  to  the 
largest  values  of  s^ 


Preliminary  experimentation  has  indicated  that  the  selection  rules 
based  on  subproblem  solution  times  (rule  (ii))  and  average  gaps  between 
the  subproblems1  linear  and  integer  programming  objective  values  (rule 
(iii))  may  be  the  most  advantageous  in  practice.  Of  these  two,  rule 
(iii)  appeared  to  be  more  consistently  effective.  However,  inasmuch  as 
the  subproblems  in  the  test  problems  run  were  all  of  equal  size,  it  is 
conceivable  that  rule  (ii)  may  indeed  be  more  favorable  when  the  differ- 


ences in  subproblem  solution  times  are  more  dramatic.  Similarly,  while 
rule  (iv)  was  found  to  be  little  more  effective  than  the  rather  arbitrary 
selection  rule  (i)  (and  sometimes  much  less  effective),  it  should  be 


pointed  out  that  subproblem  linking  constraint  coefficients  were  of 
approximately  equal  magnitude  in  all  test  problems  run.  Consequently, 
the  resulting  optimal  slacks  s ^ tended  to  be  of  equal  magnitude 
for  all  subproblems. 


For  completeness,  it  is  necessary  in  Step  7 of  Phase  2 of  the  decom- 
position algorithm  to  consider  the  situation  when  the  subproblem  (S.  ) 

J0 
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is  found  to  have  an  unbounded  optimal  solution.  Hence,  let 


integral} 


Kaneko  (197*0  has  shown  that  whenever  the  entries  of  the  matrices  A 


are  rational,  the  unboundedness  of  X(b 


and  where  x 


infinite  subset  of  the  set  of  nonnegative  numbers.  Ideally,  one  would 


in  part  (iii)  of  Step  7 to  make 

(jj*  (j„>*  (jj  (jj‘ 


like  to  choose  the  endpoint  x 


each  component  of  the  slack-vector  s 


large  as  possible,  in  the  hope  of  subsequently  improving  the  allocations 


to  the  remaining  subproblems.  However,  the  difficulty  of  determining 


an  appropriate  endpoint  for  this  purpose  may  well  be 


irohibitive,  and 


for  our  purposes  any  integer- feasible  endpoint  x 


Finally,  we  remark  that  it  is  easy  to  see  that  if  l.  > 1 for  all 


1 < j < N at  the  end  of  Phase  1 in  the  multiple  linking  constraint  case 
then  the  initial  relaxed  master  problem  (RMP)  in  Phase  2 is  sufficient 


for  solving  the  block  angular  linear  programming  problem  associated 
with  (P).  Specifically,  the  first  allocation  (b^,  ...  , b^) 


obtained  in  Step  2 of  Phase  2 of  the  decomposition  algorithm  must  lead 
to  the  overall  optimal  LP-solution  to  (P)  when  each  subproblem  (S.) 


is  solved  as  a linear  program  under  the  vector  of  linking  constraint 
~ f i ) 

right-hand  sides  bv,J  , j = 1,  2,  ...  , N.  Similarly,  the  Phase  1 pro- 
cedure in  the  single  linking  constraint  case  solves  the  corresponding 
linear  programming  relaxation  of  (p),  provided  the  parametric  functions 
z.(b.)  are  not  modified  to  eliminate  non-integer  breakpoints. 

5-3  Possible  Modifications  to  the  Algorithm 


There  are  several  possible  modifications  to  the  decomposition  algorithm 
which  may  merit  further  investigation.  It  may  be  computationally  more 
efficient,  for  example,  not  to  resolve  the  Master  Problem  after  every 
subproblem  solution.^  Instead,  after  a given  subproblem  is  solved, 
the  slacks  associated  with  the  linking  constraints  can  be  arbitrarily 
reallocated  to  the  remaining  (unfixed)  subproblems.  One  possible  heur- 
istic reallocation  procedure  that  would  be  computationally  less  expensive 
is  simply  to  assign  all  slacks  to  the  next  subproblem  to  be  solved. 

The  Master  Problem  must  at  least  be  resolved  periodically,  of  course, 
to  test  for  fathoming  and  to  identify  promising  new  allocations  when 
fathoming  does  not  occur.  Consequently,  some  computational  experimen- 
tation with  various  (possibly  dynamic)  decision  rules  governing  the 
frequency  of  Master  Problem  reoptimization  would  be  necessary.  The 
decision  not  to  resolve  the  Master  Problem  at  every  node  in  the  search 
tree  inevitably  results  in  the  loss  of  fathoming  capabilities  at  the 
"skipped"  nodes,  so  that  some  of  the  computational  savings  obtained  in 
by-passing  the  Master  Problem  may  be  lost  in  examining  nodes  which  would 


— Since  the  special  structure  of  the  Master  Problem  in  the  single  linking 
constraint  case  gives  rise  to  the  efficient  solution  procedure  developed 
in  Chapter  4,  this  modification  is  perhaps  more  relevant  for  the  multiple 
linking  constraints  case. 
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otherwise  have  been  fathomed.  However,  especially  in  large  block  angular 
problems,  where  fathoming  is  likely  to  occur  only  after  many  subproblems 
have  been  solved,  the  loss  may  well  be  minimal. 

Secondly,  it  is  not  strictly  necessary  to  branch  on  the  corresponding 
allocation  variables  immediately  after  each  subproblem  is  solved.  An 
alternative  strategy  would  be  to  successively  reallocate  any  slacks 
appearing  in  the  linking  constraints  in  each  subproblem  solution  (as 
is  currently  done),  but  to  postpone  branching  until  all  sub problems 
have  been  solved,  or  until  the  current  allocation  can  be  recognized  as 
being  suboptimal.  At  this  point,  one  subproblem  may  be  chosen,  and  the 
corresponding  allocation  variables  branched  on  in  the  usual  way. 

This  approach  would  allow  greater  flexibility  in  the  order  in  which 
the  branching  variables  are  selected.  Concomitant  with  such  an  approach, 
therefore,  might  be  the  development  of  procedures  for  computing  "penalties" 
associated  with  the  branching  process.  These  penalties,  in  analogy  with 
those  used  in  standard  (nondecomposition)  branch-and-bound  algorithms, 
could  be  used  to  guide  branch  selection,  and  to  tighten  the  objective 
function  bounds  associated  with  the  stored  nodes. 
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CHAPTER  6 


i 

' 

4 


COMPUTATIONAL  RESULTS 

Evaluation  of  the  decomposition  algorithm  presented  in  Section  5.2 
was  carried  out  on  Stanford  University’s  IBM  37o/l68  computer,  using 
programs  written  in  FORTRAN  IV  by  the  author.  In  particular,  a branch - 
and-bound  procedure  for  the  general  integer  programming  problem,  and 
two  versions  of  the  algorithm  for  decomposing  block  angular  integer 
programs  (corresponding  to  the  single  and  multiple  linking  constraint 
cases,  respectively)  were  coded. 

The  branch-and-bound  code  was  written  for  use  in  generating  the 
subproblem  solutions  required  in  Step  7 of  Phase  2 of  the  decomposition 
procedures.  In  addition,  this  code  also  served  to  provide  a guideline 
for  evaluating  the  practical  effectiveness  of  the  decomposition  algorithm) 
in  that  the  test  results  obtained  by  the  latter  can  be  compared  to  those 
which  might  be  obtained  by  a nondecomposition  approach  to  the  same  problems. 
The  decision,  specifically,  to  implement  a branch-and-bound  routine  for 
this  dual  purpose  was  made  in  consideration  of  the  fact  that  branch- 
and-bound  algorithms  appear  to  be  the  most  consistently  effective  of 
the  various  types  of  general  integer  programming  procedures. 

The  branch-and-bound  routine  coded  is  based  on  the  algorithm 
suggested  by  Dakin  (1965).  Tomlin’s  (1970)  improved  penalties  are  used 
to  enhance  the  power  of  fathoming  tests  at  each  node,  and  to  guide  branch 
selection  at  unfathomed  nodes.  Specifically,  the  branching  variable 
is  always  chosen  to  be  the  variable  with  the  largest  associated  up-  or 


hi 
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down-penalty,  and  the  forward  branch  is  made  in  the  direction  opposite 
to  the  maximum  penalty.  The  node  generated  by  the  branch  in  the  same 
direction  as  the  maximum  penalty  is  added  to  the  list  to  be  examined 
later.  This  branching  strategy  was  first  adopted  by  Little,  Murty  , 

Sweeney,  and  Karel  (1963),  and  has  since  enjoyed  considerable  success. 

(See,  for  example,  Beale  and  Small  (1965).)  Stored  nodes  are  retrieved 
from  the  list  in  accordance  with  the  last-in-first-out  (LIFO)  rule. 

(See  Section  2.2.)  For  mere  details,  the  interested  reader  is  referred 
to  the  references  above,  and  also  to  Beale  (1968). 

The  primal- simplex  code  LPM-1  was  used  to  form  the  linear  programming 
foundation  for  our  three  programs.  This  code  uses  the  product-form  of 
the  inverse,  and  was  developed  by  J.  A.  Tomlin  of  the  Systems  Optimi- 
zation Laboratory,  Department  of  Operations  Research,  Stanford  University. 
In  order  to  use  LPM-1  within  the  context  of  a branch-and-bound  procedure, 
the  program  was  modified  to  handle  variables  with  general  upper  and 
lower  bounds,  and  appropriate  row  and  column  selection  subroutines  were 
added  to  facilitate  operation  as  a dual-simplex  method  as  well.  Reopti- 
mization following  branch  in  the  forward  direction  is  carried  our  by 
dual-simplex  pivots,  while  primal- simplex  pivoting  is  used  for  reopti- 
mization when  backtracking  to  examine  a previously  stored  node.  The 
LPM-1  code  is  also  used  to  solve  the  relaxed  master  problem  (RMP)  in 
the  decomposition  program  written  for  the  multiple  linking  constraints 
case.  The  fact  that  some  of  the  variables  in  (RMP)  are  not  sign  restricted 
presents  no  additional  difficulties,  since  LPM-1  has  been  modified  to 
treat  variables  with  general  upper  and  lower  bounds. 

The  remaining  portions  of  the  codes  were  written  in  modular  form 
according  to  the  various  tasks  to  be  performed  (e.g.,  branching,  penalty 
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calculation,  fathoming  tests,  etc,,  for  the  branch-and-bound  code;  master 
problem  construction,  master  problem  solution,  branching  on  the  allo- 
cation variables,  etc.,  for  the  decomposition  codes).  The  final  version 
of  the  branch-and-bound  code  was  compiled  under  the  IBM  FORTRAN-H-Extended 
compiler,  using  the  optimization  option  at  level  two.  While  dimensioned 
to  treat  general  integer  programming  problems  with  as  many  as  2000  non- 
zero constraint  coefficients,  the  object  program  is  approximately  113  K 
bytes  in  length.  The  similarly  compiled  final  version  of  the  decompo- 
sition procedure  in  the  single  linking  constraint  case,  dimensioned  to 
handle  up  to  ten  subproblems  with  a total  of  as  many  as  1000  nonzero 
constraint  coefficients,  is  approximately  125  K bytes  in  length.  For 
both  programs,  all  instructions  and  data  are  in-core.  While  a code  for 
the  decomposition  of  block  angular  integer  programs  with  multiple  linking 
constraints  was  developed  and  debugged  under  the  WATFIV  compiler,  no 
object  code  was  generated  for  this  program.  The  test  results  reported 
below  are  for  the  single  linking  constraint  case  only. 

While  the  author  does  not  claim  the  expertise  of  a professional 
programmer,  it  is  hoped  that  by  writing  these  codes  himself,  and  by 


using  the  same  branch-and-bound  integer  programming  procedure  throughout, 
the  effects  of  discrepancies  in  programming  sophistication  on  test  results 
will  be  mitigated.  Nonetheless,  the  results  reported  here  should  not 
necessarily  be  considered  representative  of  those  which  might  be  obtained 
by  professionally  developed  codes. 


6.1  The  Testing  Procedure 

There  are  several  parameters  which  can  affect  the  performance  of 
the  decomposition  algorithm,  including  linking  constraint  tightness, 
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overall  problem  size,  subproblem  size,  the  number  of  linking  constraints 


etc.  As  limitations  on  time  and  computer  funds  prohibited  the  detailed 


examination  of  all  such  parameters,  however,  it  was  decided  to  restrict 


the  computational  testing  of  the  algorithm  primarily  to  study  the  effects 


of  linking  constraint  tightness  and  overall  problem  size 


In  order  to  test  the  effects  of  linking  constraint  tightness  on 


solution  times,  some  control  over  the  tightness  of  all  the  constraints 


must  be  established.  To  accomplish  this,  in  each  test  problem  the 


coefficients  in  each  subproblem  constraint  row  were  summed,  and  the  right' 


hand  side  (RHS)  for  that  row  was  set  to  a given  multiple  of  the  stun 


Thus,  in  the  notation  used  in  the  following  sections,  RHS 


indicates  that  the  right-hand  side  in  each  of  the  subproblem  constraints 
was  set  equal  to  3 (5)  times  the  sum  of  the  coefficients  in  the  corres- 


ponding row 


Once  the  right-hand  sides  in  the  subproblem  constraints  were  fixed. 


the  right-hand  side  for  the  linking  constraint  was  similarly  set  equal 


to  a given  multiple  of  the  sum  of  linking  constraint  coefficients.  By 


varying  this  multiple  with  RHS  fixed,  the  relative  tightness  of  the 


linking  constraint  can  be  varied 


Definition:  Let  the  right-hand  sides  of  the  subproblem  constraints  be 


fixed  at  a £iven  multiple,  say  k 


of  the  sum  of  constraint  coeffi 


cients.  Then,  if  the  right-hand  side  in  the  linking  constraint  is  set 


to  k^  times  the  sum  of  linking  constraint  coefficients,  the  tightness 


ratio  TR  is 


z-u  -XTC..  Zw.r-i.  -.-  J 


The  tightness  ratio  TR  was  used  as  a measure  of  the  degree  of 
linking  constraint  tightness  in  the  computational  results  reported  below; 
the  smaller  the  value  of  TR,  the  tighter  is  the  linking  constraint 
relative  to  the  subproblem  constraints.  Reasoning  very  intuitively,  one 
might  expect  that  the  performance  of  the  decomposition  algorithm  would 
improve  relative  to  that  of  a nondecomposition  approach  as  TR  is  in- 
creased. If  the  linking  constraint  is  so  tight  that  the  subproblem 
constraints  are  redundant,  the  problem  (p)  is  not  really  a block  angular 
integer  program  at  all,  but  in  effect  has  been  tricked  into  taking  on 
a block  angular  appearance  by  the  addition  of  redundant  subproblems. 
Consequently,  there  would  seem  to  be  little  advantage  in  using  the 
decomposition  procedure.  At  the  other  extreme,  if  TR  is  so  large 
that  the  linking  constraint  is  redundant,  (P)  consists  of  N independent 
problems  which  have  been  artificially  forced  together  into  one.  In 
this  case,  the  decomposition  approach  should  be  highly  efficient!  The 
purpose  in  examining  the  effects  of  linking  constraint  tightness,  there- 
fore, was  to  test  the  validity  of  this  conjecture,  and,  if  valid,  to 
try  to  distinguish  the  ranges  of  TR  in  which  each  of  the  two  approaches 
is  superior. 

The  method  used  to  test  the  effects  of  overall  problem  size  on 
solution  times  is  similar  to  the  method  reported  by  Reardon  (197*0* 

Once  a given  block  angular  integer  program  (P)  was  formulated  (with,  say, 
N subproblems  in  all),  derivative  problems  with  2,  4,  6,  ...  , N sub- 
problems were  obtained  by  deleting  various  numbers  of  the  subproblems  of 
(P).  Consequently,  the  derivative  problems  retained  many  of  the  char- 
acteristics of  the  source  problem,  and  so  were  closely  related  in  terms 
of  their  degree  of  computational  difficulty.  As  Reardon  pointed  out, 


■ 


ii 
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this  relationship  enables  the  direct  comparison  of  solution  times  for 
problems  of  increasing  size,  since  differences  in  solution  times  may- 
be attributed  largely  to  the  si^e  differences  alone. 

In  all,  four  test  problems  and  their  derivatives  were  solved.  The 
data  for  the  first  two  problems  was  obtained  from  a test  problem  of 
Reardon’s,  and  attention  was  concentrated  on  the  effects  on  solution 
times  of  various  levels  of  linking  constraint  tightness  in  these  problems. 
The  data  for  the  third  test  problem  was  taken  from  a capital  budgeting 
application  reported  by  Fleisher  (1976).  For  this  problem,  both  the 
effects  of  linking  constraint  tightness  and  problem  size  were  investi- 
gated in  turn.  In  the  fourth  problem,  the  data  originated  from  an  R & D 
project  selection  problem  given  by  Petersen  (1967).  The  primary  objective 
in  this  case  was  to  observe  the  dependence  of  solution  times  on  problem 
size.  In  addition,  for  both  the  third  and  fourth  test  problems,  internal 
timers  were  added  to  the  two  codes  to  monitor  the  progress  of  the  solution 
with  computation  time.  In  particular,  whenever  a new  incumbent  solution 
was  found,  the  objective  value  of  the  solution  and  the  computation  time 
required  to  reach  that  solution  were  printed  out.  This  information 
provides  insight  into  the  effectiveness  of  the  decomposition  algorithm 
as  a heuristic  procedure  for  quickly  generating  good,  though  possibly 
nonoptima 1,  solutions  for  block  angular  problems  which  may  be  too  large 
to  solve  optimally. 

6.2  Test  Problems  1 and  2 

The  first  two  test  problems  consisted  essentially  of  subproblems 
one  through  four  and  subproblems  five  through  eight,  respectively,  of 
Reardon's  test  problem  number  one.  The  original  data  for  this  problem 
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was  derived  from  Petersen’s  R & D project  selection  problem  5.  Reardon 
transformed  the  data  appropriately  to  obtain  an  integer  programming 
problem  with  the  dual  angular  structure.  By  taking  the  transpose  of  the 
first  linking  column  in  Reardon's  problem  as  the  single  linking  constraint 
in  ours,  and  dropping  the  remaining  linking  columns,  we  obtained  a problem 
with  the  block  angular  structure  suitable  for  use  in  testing  the  decompo- 
sition algorithm.  Since  the  primary  interest  initially  was  in  the  per- 
formance of  the  decomposition  algorithm  in  relation  to  the  degree  of 
linking  constraint  tightness,  rather  than  problem  size,  the  problem  was 
arbitrarily  divided  into  the  two  smaller  test  problems  of  four  subproblems 
each.  The  savings  in  computation  time  which  resulted  from  this  reduction 
enabled  more  extensive  testing  of  the  effects  of  linking  constraint 
tightness  than  would  otherwise  have  been  possible. 

Inasmuch  as  each  of  the  subproblem  constraint  matrices  consisted 
of  five  rows  and  five  columns,  both  Test  Problems  1 and  2 were  21-row, 
20-variable  integer  programs  overall.  These  problems  are  exhibited  in 
Tables  A- I and  A-II,  respectively,  in  the  appendix. 

In  Test  Problem  1,  the  right-hand  sides  of  the  subproblem  constraints 
were  first  fixed  at  three  times  the  sum  of  the  corresponding  row  coeffi- 
cients (RHS  = 300$) . Attempts  were  made  to  solve  the  problem,  both  by 
the  decomposition  algorithm  and  by  the  standard  branch-and-bound  algorithm, 
for  TR  = 0.33,  1.00,  I.67.  These  test  results  are  recorded  in  Table  I, 
and  are  plotted  in  Figure  1.  The  right-hand  sides  of  the  subproblem 
constraints  were  next  set  equal  to  RHS  = 500$,  and  the  problem  was 
rerun  with  TR  = 0.2,  0.4,  0.6,  0.8.  The  solution  times  for  these  runs 
are  shown  in  Table  II,  and  are  plotted  in  Figure  2. 


DSLC  = Decomposition  algorithm  for  single  linking  constraint  case 


BB  = Branch-and-bound  algorithm 


TABLE  II 

Test  Problem  1:  RHS  = 500% 

SOLUTION  TIMES  AS  A FUNCTION  OF  LINKING  CONSTRAINT  TIGHTNESS 


4 

E I 


TIGHTNESS 

RATIO 

Algorithm 

0.2 

0.4 

0.6 

0.8 

DSLC 

1-79 

4.88 

4.44 

> 20 

BB 

0-58 

7-25 

8.67 

> 20 

— The  two  cases TR  = 1.00,  TR  = 1.67  were  attempted  on  the  same  run, 
with  a combined  time  limit  of  30  seconds  for  the  job.  WATFIV  compilation 
required  1. 16  seconds. 


Test  Problem  2 was  treated  in  much  the  same  way  as  Problem  1.  The 
results  obtained  under  various  levels  of  linking  constraint  tightness  for 
the  two  cases,  RHS  = 300$  and  RHS  = 500$,  are  presented  in  Table  III 
and  Figure  3>  and  in  Table  IV  and  Figure  4,  respectively.  All  times  shown 
for  both  test  problems  represent  the  total  time  during  which  the  IBM 
370/l68  central  processing  unit  was  available  to  the  core  partition 
containing  the  job.  Since  these  two  test  problems  were  run  under  object 
code  produced  by  WATFIV  compiler,  the  solution  times  shown  may  be  much 
greater  (typically  by  a factor  of  about  seven)  than  those  which  would 
have  been  obtained  had  the  FORTRAN-H-Extended  compiler  been  used.  This 
presents  no  difficulty  for  our  purposes,  however,  since  it  is  the  relation- 
ships between  these  solution  times,  rather  than  the  magnitude  of  the 
times  themselves,  that  is  of  key  importance. 

6.3  Test  Problem  3 

Our  third  test  problem  was  derived  from  a 20-variable,  10-constraint 
capital  budgeting  problem  reported  by  Fleisher  (197 6).  Following  a pro- 
cedure similar  to  that  used  by  Reardon,  the  data  in  Fleisher ’s  problem 
was  transformed  so  as  to  yield  a problem  with  the  block  angular  structure. 
First,  the  twenty  projects  in  the  original  problem  were  divided  into 
forty  projects,  governed  by  five,  rather  than  ten,  constraint  rows. 

That  is,  the  first  five  rows  of  the  constraint  matrix  were  taken  to 
represent  one  group  of  twenty  projects,  and  the  last  five  rows  represented 
a second  group.  The  original  objective  coefficients  were  retained  for 
each  of  the  revised  projects;  e.g.,  since  both  columns  one  and  two  in 
our  40-variable  problem  were  obtained  from  column  one  of  the  original 
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TABLE  III 

Test  Problem  2:  RHS  = 300$ 

SOLUTION  TIMES  AS  A FUNCTION  OF  LINKING  CONSTRAINT  TIGHTNESS 


TIGHTNESS 

RATIO 

Algorithm 

0.33 

0.67 

DSLC 

3.84 

9.89 

BB 

2.13 

12.87 

TABLE  TV 

Test  Problem  2:  RHS  = 500$ 

SOLUTION  TIMES  AS  A FUNCTION  OF  LINKING  CONSTRAINT  TIGHTNESS 


— 


TIGHTNESS  RATIO 


Figure  U;  Solution  times  for  Test  Problem  2 (RHS  = 50056) 


problem,  each  was  assigned  the  cost  coefficient  from  column  one  of  the 
original  problem. 

The  forty  revised  projects  were  next  grouped  into  eight  subproblems 
of  five  variables  and  five  constraint  rows  each.  The  coefficients  in 
the  last  row  of  each  subproblem  were  then  removed  and  used  to  form  the 
linking  constraint  instead.  The  resulting  block  angular  integer  program 
thus  had  four  rows  and  five  columns  in  each  of  eight  subproblems,  and 
one  linking  constraint.  The  coefficients  defining  this  problem  are 
given  in  Table  A-III  in  the  appendix. 

The  performance  of  the  decomposition  algorithm  in  relation  to  linking 
constraint  tightness  was  tested  on  the  derivative  problem  consisting  of 
the  first  four  subproblems  of  Test  Problem  3.  The  right-hand  sides  in 
the  subproblem  constraints  were  set  to  three  times  the  sum  of  the  corres- 
ponding row  coefficients,  and  four  different  values  for  the  tightness 
ratio  TR  were  tried  in  turn.  The  results  obtained  from  both  the 
decomposition  and  the  branch-and-bound  codes  are  presented  in  Table  V ; 

and  Figure  5* 

TABLE  V 

Test  Problem  3:  RHS  = 300$ 

SOLUTION  TIMES  AS  A FUNCTION  OF  LINKING  CONSTRAINT  TIGHTNESS 


TIGHTNESS 

RATIO 

Algorithm 

0.33 

O.67 

1.00 

1.33 

DSLC 

1.74 

5.03 

8.24 

6.83 

BB 

0.83 

4.34 

15.28 

23.54 

i 
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From  these  results,  it  appeared  that  while  the  standard  branch-and- 
bound  algorithm  enjoyed  a slight  advantage  over  the  decomposition  approach 
for  small  values  of  TR,  the  advantage  turned  in  favor  of  the  latter  as 
TR  increased.  Since'  the  turning  point  was  observed  to  be  approximately 
TR  = O.67,  it  was  decided  to  fix  TR  at  this  level  in  examining  the 
effects  on  solution  times  of  increasing  problem  size.  Thus,  with  RHS  = 
300#  again  in  each  subproblem,  and  TR  = O.67  always,  the  two  additional 
derivative  problems  consisting,  respectively,  of  the  first  two  and  the 
first  six  subproblems  of  Test  Problem  3 were  solved.  (The  four-subproblem 
derivative,  of  course,  had  been  solved  previously.)  The  results  are 
shown  in  Table  VI,  and  are  plotted  in  Figure  6.  In  view  of  the  unexpec- 
tedly large  increase  in  solution  times  observed  in  increasing  the  overall 
problem  size  from  four  to  six  subproblems,  it  was  decided  not  to  solve 
the  eight- subproblem  case. 

In  each  of  the  six  versions  of  Test  Problem  3 that  were  solved, 
the  quality  of  the  successive  incumbent  solutions  obtained  by  each  of 

TABLE  VI 

Test  Problem  3:  TR  = O.67 

SOLUTION  TIMES  AS  A FUNCTION  OF  PROBLEM  SIZE 


NUMBER 

OF  SUBPROBLEMS 

Algorithm 

2 

4 

6 

DSLC 

1.34 

5.03 

26.60 

BB 

0.78 

4.34 

53.31 

EX 
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the  two  codes  was  monitored  as  described  in  Section  6.1.  The  results 

are  shown  in  Tables  VII-XII,  respectively.  Two  measures  of  the  "quality" 

of  a nonoptimal  solution  were  computed,  as  indicated  in  the  tables.  In 

# * 

general,  the  percent  error  of  a given  incumbent  solution  (x^  , ...,x^  ) 
for  any  N-subproblem  derivative  of  Test  Problem  3 is  defined  by 


N 


e = 


E 

j-i 


N 

:£ 


c(d)x(d) 


N 

£ 

3=1 


100$ 


(i) 


A/.  \ ^ 

where  (x'  , ...  , x'*')  represents  the  optimal  solution  to  the  given 

problem.  Similarly,  following  Hillier  (1969),  the  normalized  deviation 

* * 

from  optimality  for  (x^  , ...  , x^  ) is  defined  by 


d = 


£ JKw  - l 

3=1  .1=1 

V j=l 


(2) 


It  is  easy  to  show  that  d represents  the  Euclidean  distance  in  R 


N 


(n  = £ n ) from  the  point  (x 

3=1  2 


(1 ) 


.(n) 


) to  the  hyperplane 


given  by 


f . e e(d)x(d). 


Of  the  two,  therefore,  the  normal- 


3=1  3=1 

ized  deviation  from  optimality  represents  a less  problem-dependent  measure 
of  the  quality  of  a nonoptimal  solution.  The  normalized  deviations  from 
optimality  of  the  successive  incumbent  solutions  obtained  by  the  two  codes 
are  plotted  as  a function  of  computation  time  for  the  six  derivative 
problems  in  Figures  7-12- 
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TABLE  VII 

Test  Problem  3:  Four-Subproblem  Derivative,  TR  = 0.33 

QUALITY  OF  INCUMBENT  SOLUTIONS 

DSLC 


| 

’ 1 


Objective 

Norm.  Dev. 

Time 

Value 

$ Error 

from  Opt. 

0.28 

10397 

1.75 

.1885 

0.39 

10582 

0.00 

.0000 

I 


Time 

Objective 

Value 

BB 

% Error 

Norm.  Dev. 
from  Opt. 

0.40 

10505 

0.75 

.0784 

0.56 

10539 

0.^1 

.0438 

0.65 

10575 

0.09 

.0091 

O.67 

10582 

0.00 

.0000 
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TABLE  VIII 

Test  Problem  3:  Four-Subproblem  Derivative,  TR  = 1.00 

QUALITY  OF  INCUMBENT  SOLUTIONS 


Time 

Objective 

Value 

Error 

Norm.  Dev. 
from  Opt. 

Time 

Objective 

Value 

Error 

Norm . Dev, 
from  Opt. 

0.46 

14 148 

0.84 

.1222 

0.75 

12733 

IO.76 

1.5643 

2.02 

14150 

0.83 

.1202 

0.75 

13757 

3.58 

.5207 

5.17 

14 161 

0.75 

.1090 

O.76 

13791 

3.34 

.4861 

5-32 

14187 

0.57 

.0825 

0.77 

13859 

2.87 

.4168 

5.51 

14215 

0.37 

.0540 

0.78 

13905 

2.54 

.3699 

6.18 

14229 

0.27 

• 0397 

0.78 

13973 

2.07 

. 3006 

6.47 

14260 

0.06 

.0081 

0.97 

14002 

1.87 

.2710 

7.62 

14268 

0.00 

.0000 

1.02 

14019 

1-75 

• 2537 

1.11 

14058 

1.47 

.2140 

1.88 

14078 

1.33 

.1936 

3.21 

14 079 

1.32 

.1926 

3.22 

14147 

0.85 

• 1233 

4.83 

14190 

0.55 

.0794 

5.25 

14191 

0.54 

.0784 

5.41 

14199 

0.48 

• 0703 

Mjjj 
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NORMALIZED  DEVIATION  FROM  OPTIMALITY  NORMALIZED  DEVIATION  FROM  OPTIMALITY 


COMPUTATION  TIME  (SEC.) 


Figure  7:  Quality  of  incumbent  solutions.  Test  Problem  3 


2 1+  6 8 10  12 
COMPUTATION  TIME  (SEC. ) 


Figure  8: 


Quality  of  incumbent  solutions,  Test  Problem 
(four-subproblem  derivative,  TR  = 1.00) 
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TABLE  IX 


Test  Problem  3:  Four-Subproblem  Derviative,  TR  = 1.33 


QUALITY  OF  INCUMBENT  SOLUTIONS 


Objective 
Time  Value 


Norm.  Dev 
Error  from  Opt. 


Objective  Norm.  Dev 

Time  Value  % Error  from  Opt. 


0.73  14526 

0.75  14594 

0.78  14662 

O.87  14708 

0.94  14725 

1.03  14747 

1.25  14760 

I.27  14774 

1.37  14185 

1.68  14189 

1.76  14843 

5.04  14862 

5.43  14921 

7.62  14964 

10.83  14990 

10. 87  15007 

17-40  15029 

17.58  15033 


0000 


TABLE  X 
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Test  Problem  3: 


Two-Subproblem  Derivative, 


TR  = 0.67 


QUALITY  OF  INCUMBENT  SOLUTIONS 


DSLC 

BB 

Time 

Objective 

Value 

$ Error 

Norm.  Dev. 
from  Opt. 

Time 

Objective 

Value 

$ Error 

Norm.  Dev, 
from  Opt. 

0.77 

64  36 

0.97 

.0899 

0.19 

6l4l 

5.51 

.5112 

0-79 

64  99 

0.00 

.0000 

0.20 

6209 

4.46 

.4l4l 

0.21 

6255 

3-75 

.3484 

0.23 

6340 

2.45 

.2270 

0.31 

6352 

2.26 

.2099 

0.36 

6369 

2.00 

.1856 

0.47 

6377 

1.88 

.1742 

0.48 

6445 

0.83 

.0771 

0.68 

6482 

0.26 

.0242 

0.71 

6499 

0.00 

.0000 

TABLE  XI 

Test  Problem  3:  Four-Subproblem  Derivative,  TR  = O.67 

QUALITY  OF  INCUMBENT  SOLUTIONS 


DSLC  BB 


Time 

Objective 

Value 

% Error 

Norm.  Dev. 
from  Opt. 

Time 

Objective 

Value 

<&  Error 

Norm.  Dev. 
from  Opt. 

0.36 

13083 

0.62 

.0835 

0.55 

13034 

1.00 

• 1335 

1.65 

13137 

0.21 

.0285 

0.6l 

13057 

0.82 

.1100 

1.72 

13143 

0.17 

.0224 

0.73 

13077 

O.67 

.0896 

1.83 

13157 

0.06 

.0081 

0.76 

13151 

0.11 

.0142 

3.70 

13165 

0.00 

.0000 

1.22 

13165 

0.00 

.0000 
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Figure  9:  Quality  of  incumbent  solu 
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Figure  10:  Quality  of  incumbent  solutions,  Test  Problem  3 

(two-subproblem  derivative,  TR  = O.67) 
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TABLE  XII 

Test  Problem  l:  Six-Subproblem  Derivative,  TR  = O.67 

.'.'ALITY  OF  INCUMBENT  SOLUTIONS 


)SLC  BB 


Objective 

1 trro. 

Norm.  Dev. 
t raa  Opt. 

Time 

Objective 

Value 

$ Error 

Norm.  Dev, 
from  Opt. 

. 

* 

A • “v» 

.2302 

1.29 

16604 

8.71 

1.3764 

1.21  17986 

1 « X a 

.1755 

1.30 

16938 

6.87 

1.0862 

2.5)  i8057 

0 8^ 

.1312 

1.32 

17246 

5.18 

1 r\ 

3 

co 

O An  1 A 1 C 1 

A Of) 

.0321 

1.35 

17791 

2.18 

.3449 

k.17  l8l86 

0.  DO 

.0000 

1.35 

17834 

1.95 

.3076 

1.35 

17877 

1.71 

.2702 

1.36 

18099 

0.49 

• 0775 

1.37 

18119 

0.38 

• 0599 

2.04 

18173 

0.08 

.0130 

4.36 

18188 

0.00 

.0000 
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6.4  Test  Problem  4 


I 

i 
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The  data  for  the  fourth  test  problem  was  taken  from  the  seventh 
of  the  R & D project  selection  problems  reported  by  Petersen  (1967). 

To  obtain  a problem  with  the  block  angular  structure,  Petersen's  50- 
variable,  5-constraint  problem  was  divided  into  ten  subproblems  of  five 
columns  and  five  rows  each.  The  coefficients  in  the  first  constraint 
in  each  subproblem  were  then  removed  and  used  to  form  the  linking  con- 
straint instead.  Finally,  each  of  the  objective  coefficients  was  divided 
by  ten,  and  rounded  to  the  nearest  integer.  Test  Problem  4 is  exhibited 
in  Table  A- IV  in  the  appendix. 

Because  the  decision  variables  in  many  practical  integer  programming 
problems  are  restricted  to  take  on  only  the  values  zero  or  one,  it  was 
felt  that  the  testing  of  the  decomposition  algorithm  should  include  at 
least  one  such  problem.  Consequently,  in  each  of  the  (five)  derivative 
problems  of  Test  Problem  4,  unit  upper  bounds  were  specified  on  all 
variables.  The  subproblem  constraint  right-hand  sides  were  fixed  at 
(approximately)  RHS  = 70$.  (This  was  the  setting  used  in  the  zero-one 
test  problems  given  by  Reardon  (1974).)  Correspondingly,  the  linking 
constraint  right-hand  side  was  chosen  always  to  be  approximately  4o$ 
of  the  sum  of  the  linking  constraint  coefficients,  so  that  TR  r:  0.57 
in  each  of  the  derivative  problems. 

The  five  derivative  problems  solved  consisted,  respectively,  of  the 
first  2,  4,  6,  8 and  10  (all)  of  the  subproblems  of  Test  Problem  4.  The 
total  solution  times  required  by  the  decomposition  and  the  standard 
branch-and-bound  algorithms  are  given  in  Table  XIII,  and  these  values 
are  plotted  in  Figure  13.  Although  the  decomposition  procedure  had 

125 


i 

i 


1 


i 


4 


•*] 


1 


located  the  optimal  solution  for  the  last  derivative  problem  in  less  than 
51  seconds,  it  had  still  failed  to  verify  the  optimality  of  this  solution 
after  60  seconds  (total)  of  CPU  usage,  and  computations  were  terminated 
prematurely. 

As  with  Test  Problem  3,  the  quality  of  the  incumbent  solutions 
produced  by  each  of  the  two  codes  was  observed  in  each  of  the  five  deri- 
vative problems.  With  "percent  error"  and  "normalized  deviation  from 
optimality"  again  defined  by  (l),  (2),  these  intermediate  results  are 
shown  in  Tables  XIV-XVIII.  The  normalized  deviation  from  optimality  of 
the  successive  incumbent  solutions  is  plotted  as  a function  of  computation 
time  in  Figures  14-18. 


6.5  Summary  of  Computational  Results 

The  results  obtained  in  the  first  three  test  problems  (specifically, 
those  plotted  in  Figures  1-4)  support  the  conjecture  made  in  Section  6.1; 
for  these  problems,  the  decomposition  algorithm  has  enjoyed  its  greatest 
computational  advantage  over  the  nondecomposition  approach  for  the  larger 
test  values  of  the  tightness  ratio  TR.  In  particular,  in  our  limited 
experimental  sample,  the  ratio  TR  = O.67  emerged  as  the  level  of  linking 
constraint  tightness  above  which  the  decomposition  procedure  was  con- 
sistently and  significantly  more  effective  than  the  standard  branch-and- 
bound  algorithm.  However,  the  solution  times  required  by  the  decomposition 
method  were  always  competitive  with,  and  often  superior  to,  those  required 
by  the  branch-and-bound  algorithm  for  tightness  ratios  even  as  low  as 
TR  = 0.2. 

The  results  of  experiments  with  increasing  problem  size  in  Test 


Problems  3 and  4 were  less  conclusive.  Figures  6 and  13  indicate  that 
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TABLE  XIII 


Test  Problem  4:  TR  = 0.57 


SOLUTION  TIMES  AS  A FUNCTION  OF  PROBLEM  SIZE 


DSLC 


TABLE  XIV 


Test  Problem  4:  Two -Sub problem  Derivative,  TR  = 0.57 


QUALITY  OF  INCUMBENT  SOLUTIONS 


Objective 

Value 


Objective  Norm.  Dev 

Time  Value  Error  from  Opt. 


Norm.  Dev 


NUMBER  OF  SUBPROBLEMS 

Algorithm 

2 4 6 8 10 

NUMBER  OF  SUBPROBLEMS 


Figure  13:  Solution  times  for  Test  Problem  4 (TR  = 0.57 


COMPUTATION  TIME  (SEC.) 


Figure  14 : Quality  of  incumbent  solutions,  Test  Problem  4 

(two-subproblem  derivative,  TR  = 0.57) 
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TABLE  XV 

Test  Problem  4:  Four -Sub problem  Derivative,  TR  = 0-57 

QUALITY  OF  INCUMBENT  SOLUTIONS 


Time 

Objective 

Value 

$ Error 

Norm.  Dev. 
from  Opt. 

Time 

Objective 

Value 

<io  Error 

Norm.  Dev 
from  Opt. 

0.29 

442 

4.55 

.0596 

0.44 

4l4 

10.59 

.0951 

0.57 

450 

2. 60 

.0257 

0.46 

451 

6.71 

.06l4 

0.47 

452 

2.16 

.0198 

0.51 

445 

5.68 

.0556 

0.47 

455 

1.52 

.0158 

0.51 

462 

0.00 

.0000 

0.57 

457 

1. 08 

.0099 

0.60 

462 

0.00 

.0000 

TABLE  XVI 

Test  Problem  4:  Six-Subproblem  Derivative,  TR  = 0.57 

QUALITY  OF  INCUMBENT  SOLUTIONS 


Time 

Objective 

Value 

$ Error 

Norm . Dev . 
from  Opt. 

0.4o 

557 

5-9^ 

.0450 

0.75 

541 

5.22 

.0552 

0.74 

544 

2.68 

.0295 

1.19 

545 

2.50 

.0275 

1.21 

546 

2.52 

.0254 

1.27 

552 

1.25 

.0156 

1.37 

555 

1.07 

.0117 

1.58 

554 

0.89 

.0097 

1.48 

556 

0.54 

.0058 

1.51 

559 

0.00 

.0000 

Objective 
Time  Value 


Norm.  Dev. 
Error  from  Opt. 

0.00  .0000 
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COMPUTATION  TIME  (SEC.) 

Quality  of  incumbent  solutions,  Test  Problem  4 
(four-subproblem  derivative,  TR  = 0.57) 


DSLC 


COMPUTATION  TIME  (SEC.) 


Figure  16:  Quality  of  incumbent  solutions,  Test  Problem  4 

(six-subproblem  derivative,  TR  = 0.57) 
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TABLE  XVII 

Test  Problem  4;  Eight-Subproblem  Derivative,  TR  = 0.57 
QUALITY  OF  INCUMBENT  SOLUTIONS 


Time 

Objective 

Value 

$ Error 

Norm.  Dev. 
from  Opt. 

Time 

Objective 

Value 

$ Error 

Norm.  Dev. 
from  Opt. 

0.52 

667 

2.78 

.0365 

1.85 

675 

1.60 

.0211 

0.80 

670 

2.33 

.0307 

1-95 

677 

1.31 

• 0173 

1.08 

672 

2. 04 

.0269 

1.96 

680 

O.87 

.0115 

1.51 

674 

1-75 

.0230 

2.72 

681 

0.73 

.0096 

1.64 

675 

1.60 

.0211 

2.85 

684 

0.29 

.0038 

I.72 

676 

1.46 

.0192 

5.56 

686 

0.00 

.0000 

1.76 

677 

1.31 

• 0173 

1.79 

679 

1.02 

.0134 

1.90 

680 

0.87 

.0115 

1.94 

682 

0.58 

.0076 

3.07 

683 

0.44 

.0057 

3-19 

686 

0.00 

.0000 
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TABLE  XVIII 


Test  Problem  4;  . Ten-Subproblem  Derivative,  TR  = 0.57 


QUALITY  OF  INCUMBENT  SOLUTIONS 


Objective  Norm  Dev 

Time  Value  $ Error  from  Opt 


Objective 
Time  Value 


Norm.  Dev 
from  Opt. 


1114 

1036 

1020 
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for  small  problems  (few  subproblems),  solution  times  with  the  decomposition 
method  are  comparable  to  those  obtained  with  the  branch-and-bound  procedure. 
As  overall  problem  size  was  increased  (by  adding  more  subproblems,  each 
of  the  same  size),  both  methods  exhibited  what  appears  to  be  an  exponential 
growth  in  the  corresponding  solution  times.  Nonetheless,  the  results 
obtained  with  the  decomposition  procedure  usually  represented  a marked 
computational  savings  in  the  larger  test  problems. 

The  notable  exception  to  this  rule,  however,  was  the  ten-subproblem 
derivative  of  Test  Problem  4,  where  the  decomposition  algorithm  had  still 
failed  to  terminate  in  over  twice  the  total  solution  time  required  by 
the  branch-and-bound  code.  One  possible  cause  for  this  result  might  be 
that  the  decision  variables  in  this  problem  were  restricted  to  take  on 
only  the  values  zero  or  one.  Since  increasing  the  number  of  subproblems 
also  increases  the  number  of  allocation  variables  requires  for  decompo- 
sition, it  may  be  that  the  number  of  combinations  of  allocation  variables 
which  must  be  examined  in  the  decomposition  approach  increases  at  a faster 
pace  than  does  the  number  of  combinations  of  the  original  0-1  decision 
variables  which  must  be  examined  in  the  branch-and-bound  procedure.  If 
this  is  indeed  the  case,  the  decomposition  algorithm  may  experience 
difficulties  with  large  block  angular  zero-one  integer  programs. 

However,  since  the  gap  in  solution  times  in  Test  Problem  4 widened 
in  favor  of  the  decomposition  method  as  the  number  of  subproblems  was 
gradually  increased  from  two  to  eight,  the  result  in  the  ten-subproblem 
derivative  is  not  necessarily  representative  of  the  general  behavior 
of  the  two  algorithms.  Moreover,  the  fact  that  a tightness  ratio  of 
TR  = 0.57  was  maintained  in  the  fourth  test  problem  must  also  be 
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considered.  This  level  of  linking  constraint  tightness  is  below  the 


"critical"  value  of  TR  = 0.67  that  was  identified  in  the  first  three 
test  problems;  while  the  decomposition  method  was  consistently  superior 
to  the  branch-and-bound  algorithm  when  TR  > 0-^7,  neither  method  enjoyed 
a clear-cut  advantage  in  test  problems  with  TR  < 0.67-  In  this  light, 
the  results  in  Test  Problem  4 may  equally  well  be  considered  as  an  indi- 
cation that  the  degree  of  linking  constraint  tightness  is  a more  crucial 
factor  in  determining  the  consistency  of  the  effectiveness  of  decomposition 
than  is  problem  size.  Verification  of  this  hypothesis  in  further  compu- 
tational tests  has  important  implications  for  the  user  who  must  select 
a singu.?  method  for  a given  application,  since  the  tightness  ratio  TR 
is  a simple  measure  which  can  be  readily  approximated  from  problem  data 
alone. 

Finally,  the  intermediate  results  obtained  in  Test  Problems  3 and  4 
(see  Figures  7-12  and  14-18)  would  strongly  encourage  the  use  of  the 
decomposition  algorithm  as  a heuristic  procedure  for  block  angular  integer 
programs  which  may  be  too  large  to  solve  optimally,  or  for  which  an  exact 
optimal  solution  is  not  required.  As  a measure  of  the  effectiveness  of 
decomposition  in  quickly  identifying  good  (suboptimal)  solutions,  let 
t^  denote  the  computation  time  required  by  the  decomposition  algorithm 


to  find  a first  feasible  solution  to  a given  problem,  and  correspondingly. 


let  tg  denote  the  time  required  by  the  branch-and-bound  algorithm  to 


reach  a feasible  solution  of  the  same  quality.  In  the  various  versions 


of  Test  Problems  3 and  4 that  were  solved,  tg  was  observed  to  range 


from  2 to  43  times  t^.— ' In  particular,  in  the  two  largest  derivatives 


— The  single  exception  was  the  two-subproblem  derivative  of  Test  Problem  J . 
Because  of  its  small  size,  however,  this  problem  is  so  easily  solved  opti- 
mally that  the  use  of  a heuristic  procedure  would  be  questionable  in  any  event. 
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of  these  problems  - the  first  with  30  general  integer  variables  and  the 
second  with  50  zero-one  variables  overall  - the  decomposition  procedure 
reached  a first  feasible  solution  in  0 .64  and  0.65  seconds,  respectively. 
By  comparison,  the  branch-and-bound  algorithm  required  1.35  and  8.63 
seconds,  respectively,  to  locate  integer  feasible  solutions  of  approxi- 
mately the  same  quality.  These  results  are  especially  encouraging  in 
view  of  the  fact  that  the  corresponding  tightness  ratios  of  TR  = 0.67 
and  TR  = 0.57  were  fairly  small. 

In  general,  the  decomposition  procedure  consistently  and  quickly 
produced  initial  feasible  solutions  of  high  quality.  Moreover,  its 
advantage  over  the  branch-and-bound  algorithm  in  this  regard  increased 
with  increasing  problem  difficulty.  In  Test  Problem  4 in  particular, 
t^  was  observed  to  increase  approximately  linearly  with  increasing 
problem  size.  Recall  that  when  the  block  angular  integer  program  (P) 
is  decomposed  into  its  component  subproblems,  the  work  required  to 
generate  a first  incumbent  essentially  involves  the  solution  of  N 
parametric  linear  programs,  followed  by  the  solution  of  N integer 
programming  problems.  If  the  total  problem  size  is  doubled,  therefore, 
the  work  to  find  a first  feasible  solution  involves  not  the  solution 
of  linear  and  integer  programming  problems  of  twice  the  original  size, 
but  rather  the  solution  of  twice  as  many  linear  and  integer  programs 
of  the  same  size.  Hence,  there  is  reason  to  expect  that,  at  least  in 
the  single  linking  constraint  case,  the  observed,  linear  increase  in 
the  computation  time  required  by  the  decomposition  algorithm  to  obtain 
a first  feasible  solution  will  extend,  to  other,  larger  block  angular 
integer  programs  as  well. 
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CHAPTER  7 

CONCLUSIONS,  EXTENSIONS,  AND  AREAS  FOR  FUTURE  RESEARCH 
7.1  Conclusions 

The  computational  results  reported  in  Chapter  6 are  too  few  to  draw 
any  final  conclusions  concerning  the  performance  of  the  decomposition 
algorithm.  Nonetheless,  these  first  experiments  would  indicate  that 
our  procedure  for  the  decomposition  of  block  angular  integer  programs 
can  be  an  attractive  alternative  to  standard  integer  programming  methods 
which  ignore  problem  structure.  In  particular  the  decomposition  approach 
has  been  most  effective  in  treating  problems  in  which  the  linking  con- 
straints are  fairly  loose  relative  to  the  subproblem  constraints,"  in 
terms  of  the  measure  of  relative  linking  constraint  tightness  TR 
defined  in  Section  6.1,  solution  times  exhibited  by  the  decomposition 
algorithm  in  problems  with  TR  > O.67  were  45-70$  less  than  those 
required  by  a nondecomposition  approach  to  the  same  problems.  Thus, 
it  may  well  be  possible  to  determine  in  advance  (by  approximating  TR 
from  the  problem  data)  which  problems  can  be  most  efficiently  solved 
through  the  decomposition  approach. 

Also  encouraging  was  the  observation  that  computation  times 
required  by  the  decomposition  algorithm  usually  experienced  a significantly 
less  rapid  growth  with  increasing  problem  size  than  did  the  corresponding 
times  for  the  standard  branch-and-bound  procedure.  There  was  one  major 
exception  to  this  trend,  however,  and  more  extensive  computational  testing 
of  the  algorithm  on  other  and  larger  block  angular  problems  would  be 
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desirable.  An  investigation  of  the  effects  of  different  levels  of  linking 
constraint  tightness  on  solution  times  in  these  larger  problems  is  espe- 
cially important.  Moreover,  since  the  results  reported  in  Chapter  6 
were  obtained  with  block  angular  problems  having  only  a single  linking 
constraint,  there  is  also  the  need  to  test  the  performance  of  the  decom- 
position algorithm  in  the  multiple  linking  constraints  case. 

Finally,  our  algorithm  rapidly  and  consistently  generated  initial 
feasible  solutions  of  high  quality.  Especially  in  the  single  linking 
constraint  case,  there  is  reason  to  believe  that  the  computation  time 
required  to  reach  a first  feasible  solution  will  grow  approximately 
linearly  with  the  number  of  subproblems.  Consequently,  the  decomposition 
procedure  also  can  be  valuable  as  a heuristic  method  for  quickly  locating 
good  (suboptimal)  solutions  to  large  block  angular  integer  programs. 

7.2  Extensions  and  Areas  for  Future  Research 

It  is  hoped  that  the  present  work  will  be  regarded  as  only  a first 
step  towards  the  development  of  efficient  decomposition  procedures  for 
block  angular  integer  programming  problems,  and  as  such,  will  serve  to 
motivate  continued  and  original  efforts  in  this  area.  Several  possible 
modifications  to  the  decomposition  algorithm  already  have  been  described 

V 

in  Section  5»3*  Of  these,  the  development  of  a system  of  penalties  for 
use  in  association  with  the  branching  process  is  perhaps  of  greatest 
potential  value.  Historically,  the  introduction  of  the  penalty  concept 
in  standard  (nondecomposition)  branch-and-bound  integer  programming 
methods  has  resulted  in  significant  improvements  in  the  efficiency  of 
these  methods.  The  derivation  and  incorporation  of  penalty  calculations 
in  the  decomposition  algorithm  might  best  be  combined  with  an  investigation 
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of  alternative  branching  strategies  as  well.  One  such  alternative  was 
discussed  briefly  in  Section  5.3- 

There  is  also  the  need  for  continued  computational  analysis  of  the 
performance  of  the  decomposition  algorithm.  In  addition  to  the  problem 
parameters  investigated  in  Chapter  6,  two  others  which  may  have  important 
effects  on  algorithmic  efficiency  are  the  number  of  linking  constraints 
and  (average)  subproblem  size.  It  would  be  extremely  useful  from  a 
practical  viewpoint  to  identify  the  ranges  of  these  parameters  for  which 
the  decomposition  procedure  is  most  effective. 

The  algorithm  for  decomposing  block  angular  integer  programs  often 
can  require  that  several  different  versions  of  each  of  the  subproblems 
be  solved.  Since  these  versions  will  differ  from  one  another  only  in  the 
right-hand  sides  of  the  linking  constraints,  it  may  well  turn  out  that 
significant  computational  savings  can  be  achieved  by  incorporating 
techniques  for  solving  these  subproblems  parametrically,  or  for  conduc- 
ting sensitivity  analyses  on  the  optimal  subproblem  solutions.  Recently, 
many  advances  have  been  made  in  the  areas  of  parametric  integer  pro- 
gramming and  postoptimality  analysis  in  integer  programming.—^  Such 


techniques  may  be  particularly  valuable  for  block  angular  integer  pro- 


grams in  which  the  subproblems  themselves  are  fairly  large,  since  greater 


proportions  of  the  total  solution  time  for  these  problems  will  be  spent 
in  generating  the  subproblem  solutions. 

A closely  related  issue  is  the  question  of  optimal  degree  of  decom- 
position.  In  the  algorithm  developed  here,  each  of  the  subproblems  is 


— See,  for  example,  Geoffrion  and  Nauss  (1976),  Piper  and  Zoltners 
(1975),  (1976),  Radke  (197?);  Roodman  (1972),  and  Nauss  (1975). 
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treated  individually.  In  a sense,  the  straightforward  branch-and-bound 


* 


f 

i 


procedure  is  simply  an  alternative  version  of  the  decomposition  algorithm 
in  which  all  the  subproblems  are  combined  and  treated  as  one.  Inter- 
mediate versions  between  these  two  extremes  are  certainly  possible  as 
well.  Since  the  straightforward  branch-and-bound  algorithm  proved  slightly 
superior  to  the  decomposition  approach  in  the  two -subproblem  derivatives 
of  Test  Problems  3 and  b,  it  seems  reasonable  to  consider,  for  example, 
pairing  individual  subproblems  and  treating  each  such  combined  pair 
as  a single  subproblem  when  the  decomposition  algorithm  is  applied.  In 
general,  it  would  be  interesting  to  compare  solution  times  resulting  from 
various  different  strategies  for  grouping  the  subproblems.  Since  partial 
decomposition  approaches  necessarily  will  lead  to  increased  subproblem 
sizes,  the  incorporation  of  efficient  parametric  integer  programming 
capabilities  may  well  be  an  important  factor  in  determing  the  relative 
effectiveness  of  different  degrees  of  decomposition. 

Underlying  the  branching  procedure  in  the  decomposition  algorithm 
is  the  assumption  that  the  allocation  variables  can  take  on  only  discrete 
values.  In  the  present  work,  we  have  guaranteed  that  this  will  always 
be  the  case  by  assuming  that  1)  all  linking  constraint  coefficients  in 
the  block  angular  integer  program  (P)  are  integral,— ^ and  2)  all  decision 


: 

« 


I 


— Rational  coefficients  can  always  be  made  integral.  Moreover,  since 
digital  computers  can  represent  problem  coefficients  only  with  finitely 
many  decimal  places,  in  principle  it  is  always  possible  to  render  all 
coefficients  integral  by  scaling  appropriately.  However,  because  the 
magnitude  of  the  resulting  (integral)  coefficients  may  be  quite  large, 
this  approach  to  circumventing  assumption  l)  can  lead  to  serious  diffi- 
culties with  the  numerical  stability  of  the  algorithm. 


1U0 


variables  are  integer-restricted.  As  was  previously  mentioned,  in  certain 
applications  physical  considerations  may  naturally  imply  that  the  allo- 
cation variables  can  be  meaningfully  interpreted  only  at  discrete  levels 
(e.g.,  if  they  actually  represent  resources  which  can  be  subdivided  only 
in  integral  amounts).  In  such  cases,  our  method  is  applicable  even  when 
assumptions  l)  and  2)  are  not  satisfied.  Nonetheless,  a valuable  extension 
in  the  range  of  problems  for  which  the  decomposition  algorithm  is  suitable 
would  be  obtained  if  one  or  both  of  these  conditions  could  be  dropped. 

The  second  assumption  in  particular  rules  out  the  important  class  of 
mixed-integer  programming  problems.  Benders  (1962)  has  developed  a 
procedure  for  decomposing  general  mixed-integer  programming  problems  into 
their  integer  and  linear  programming  component  parts,  and  a similar  approach 
might  be  considered  here.  The  resulting  integer  programming  component, 
however,  must  retain  its  block  angular  structure  without  violating  the 
first  assumption  above. 

The  multi-stage  integer  program  (Q)  exhibited  below  is  quite  similar 
in  structure  to  the  block  angular  problem. 


(Q) 

maximize  c 
subject  to 


<1)x<1hc(2)x<2)+c(3)x(3)+...tc(»-l)x(N-l)tc(B)x{N)  f 


A<2)*<2W2>x<5> 


1 P 
< P 


(1) 

(2) 


.(d) 


A(N-l)x(N-l)+B(N-l)x(N)  < p(N-l) 
, N . 


> 0,  integer,  j = 1,  2,  ... 


By  properly  reordering  the  constraint  rows,  (Q)  can  be  put  into  block 


angular  form.  This  can  be  done,  for  example,  by  moving  alternate  levels 


Au'xVJ'  + (j  = 1,  3,  5,  etc.)  to  the  top  to  form 

the  linking  constraints.  The  remaining  levels  A^x^'  + B^x^+"^ 


2,  4,  6,  etc.)  then  form  the  subproblems'  constraints 


The  decomposition  algorithm  for  block  angular  integer  programs  can  thus 


be  applied  to  problem  (Q) . Some  effort  should  be  made,  however,  to 


exploit  the  special  structure  of  the  linking  constraints  themselves  in 


such  problems.  For  example,  each  subproblem  will  have  all-zero  coeffi 


cients  in  many  of  the  linking  constraints.  The  total  number  of  allo- 


cation variables  required  for  decomposition  can  be  reduced  by  noting  that 


the  allocation  of  the  i linking  resource  to  the  j subproblem  can 


obviously  be  fixed  at  zero  if  the  coefficients  corresponding  to  this 


subproblem  are  identically  zero  in  the  i linking  constraint 


Alternatively,  problem  (Q)  can  be  put  into  dual  angular  form  by 


reordering  the  decision  variables  so  that,  for  example,  the  columns 


appear  in  the  left-most 


corresponding  to  x 


positions  in  the  revised  constraint  matrix.  These  variables  then  form 


the  linking  columns,  while  the  subproblems  are  comprised  of  the  remaining 
variables.  Reardon  (1974)  has  developed  an  efficient  decomposition  method 


for  dual  angular  integer  programming  problems.  The  linking  columns  in 
the  dual  angular  problem  obtained  from  the  multi-stage  problem  (Q)  will 


again  have  a special  structure  of  their  own,  and  some  effort  should  be 


made  to  exploit  this  fact.  It  would  be  interesting  to  compare  the  per 


formance  of  Reardon's  method  with  that  of  the  algorithm  presented  here 
when  each  is  applied  to  problems  of  the  form  (q). 


L 


Finally,  the  doubly-coupled  integer  program  is  also  closely  related 
to  both  the  block  angular  and  dual  angular  integer  programming  problems. 
These  problems  are  characterized  by  a block  diagonal  matrix  of  constraint 
coefficients  in  which  both  linking  rows  and  linking  columns  are  present. 
Consequently,  the  development  of  an  algorithm  combining  Reardon's  method 
with  the  one  developed  here  would  seem  appropriate.  One  such  approach 
might  begin  by  treating  the  problem  as  a dual  angular  integer  program 
with  one  (large)  subproblem.  In  accordance  with  Reardon's  algorithm, 
the  linking  variables  are  assigned  values  first,  whereupon  decomposition 
into  independent  subproblems  occurs.  In  this  case,  of  course,  there  is 
only  a single  subproblem.  However,  this  subproblem  itself  has  a block 
angular  structure,  and  so  may  be  solved  by  the  method  developed  in  the 
present  work. 


TABLE  A- 1 
Test  Problem  1 


COLUMNS 

1 

2 

3 

4 

5 

RHS-7 

.W 

20 

20 

15 

15 

20 

8 

8 

3 

5 

5 

80 

55 

70 

4o 

80 

975 

B<l} 

100 

10 

100 

4 

90 

912 

20 

20 

20 

14 

30 

312 

40 

30 

30 

29 

4o 

507 

50 

35 

40 

24 

4o 

582 

COLUMNS 


8 8 3 5 


40  45  21  15 


10  75  0 

20  8 6 

20  16  12 

20  19  16 


5 RHS^7 


20  10  10  6 6 


9 390 

25  6 348 

3 12  147 

5 12  195 

7 15  231 


COLUMNS 

EHsi7 

1 

2 

3 

4 

5 

=<3) 

8 

8 

29 

29 

20 

a<5) 

8 

8 

3 

5 

5 

28 

18 

90 

29 

130 

885 

b(3) 

28 

0 

0 

OJ 

H 

6 

130 

852 

12 

10 

14 

18 

4o 

282 

18 

10 

24 

30 

60 

426 

18 

10 

29 

30 

70 

471 

COLUMNS 

1 2 3 4 5 RHsi7 


7 7 5 5 


8 8 3 5 5 


70  32  21  20  17  480 

32  32  3 40  0 321 


32  32  3 40  0 321 

42  6 9 3 12  216 

42  16  18  11  18  315 

42  21  20  17  18  354 


' 

Tightness  Ratio 
0 33  1.00  1.67 

Linking  Constraint 
Right-hand  Side 

116  348  580 

Tightness  Ratio 
0.2  0.4  0.6  0.8 

Linking  Constraint 
Right-hand  Side 

116  232  348  464 

- These  right-hand  sides  represent  the  RHS  = 300%  case, 
is  obtained  by  multiplying  these  values  by  5/3* 


The  RHS  = 500%  case 


TABLE  A- II 
Test  Problem  2 


COLUMNS 

1 

2 

3 

4 

5 

RHS— 

65 

65 

33 

33 

16 

8 

8 

3 

5 

5 

120 

35 

40 

25 

30 

750 

b'1’ 

160 

70 

4o 

10 

60 

1020 

20 

100 

5 

20 

0 

435 

30 

no 

25 

20 

10 

585 

30 

120 

25 

20 

15 

630 

COLUMNS 

1 2 3 4 


16  24  24  4 


8 8 


?0  20 
0 55 

5 5 


0 10 
6 3 


15  13  18 

20  25  22 


5 RHS— ^ 


5 228 

0 195 

4 69 

7 174 

7 237 


COLUMNS 

2 3 


5 RHS— ^ 


3 130  130  155 


8 3 5 5 


2 180  110  220  1545 

0 24 0 0 290  1608 

1 20  20  30  213 

2 80  40  60  549 

3 100  50  70  672 


COLUMNS 
2 3 


5 RHS- 


155  55  55  48  48 


8 8 3 5 5 


RHS  = 300$ 

Tightness  Ratio 
0.33  0.67 


Linking  Constraint 
Right-hand  Side 


11 6 232 


RHS  = 500? 


Linking  Constraint 
Right-hand  Side 


Tightness  Ratio 
0.2  0.4  0.6  0.8 


116  232  348  464 


- These  right-hand  sides  represent  the  RHS  = 300$  case.  The  RHS  = 500$  case 
is  obtained  by  multiplying  these  values  by  5/3. 
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T 

COLUMNS 
2 3 k 

5 

RHS 

24  5 

2k  5 

177 

177 

291 

11 

41 

62 

68 

43 

77 

28 

12 

85 

57 

777 

21 

32 

85 

42 

87 

801 

72 

77 

58 

18 

72 

891 

76 

53 

14 

83 

86 

936 

COLUMNS 

1 

2 3 4 

5 RHS 

on 

291 

237 

237 

114 

114 

on 

47 

56 

16 

80 

5 

3 

7 

31 

21 

93 

465 

b<2) 

51+ 

82 

90 

26 

59 

933 

87 

93 

61 

3 

64 

924 

68 

37 

50 

12 

86 

759 

COLUMNS 

COLUMNS 

1 

2 

3 

4 

5 

RHS 

1 

2 

3 

4 

5 

RHS 

6HI 

237 

237 

194 

19!+ 

211 

■B8I 

211 

231 

231 

211 

211 

oil 

52 

9 

93 

35 

38 

■Oil 

52 

89 

88 

33 

73 

20 

44 

85 

93 

52 

882 

73 

72 

35 

90 

28 

894 

61 

98 

36 

94 

1 

870 

(4) 

B' 

88 

14 

59 

19 

86 

798 

93 

37 

17 

4l 

54 

726 

39 

92 

3 

79 

18 

693 

48 

20 

5& 

19 

58 

597 

58 

32 

48 

15 

99 

756 

COLUMNS 

1 

2 

3 

4 

5 

RHS 

BUI 

97 

97 

168 

168 

174 

m 

17 

64 

32 

96 

35 

62 

11 

71 

37 

93 

822 

0(5) 

74 

10 

26 

0 

85 

585 

12 

20 

20 

55 

24 

393 

6 

57 

54 

67 

32 

648 

COLUMNS 

1 

2 

3 

4 

5 

RHS 

174 

134 

134 

308 

308 

Oil 

9 

76 

62 

51 

79 

99 

8 

73 

79 

91 

1050 

59 

24 

15 

56 

76 

690 

50 

30 

30 

99 

51 

780 

21 

32 

36 

71 

4l 

603 

J 


Four-Subproblem  Derivative 


0.33 

Tightness  Ratio 

0.67  1.0c 

1.33 

Linking  Constraint 
Right-hand  Side 

1041 

2082 

3123 

4l64 

Tightness 

Ratio  = 

O.67 

Number 

of  Subproblems 

2 

4 

6 

Linking  Constraint 
Right-hand  Side 

958 

2082 

3124 
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Decomposition  in  Integer  Programming,  Gary  A.  Kochman 

Linear  programming  models  in  which  the  constraint  matrix  has  a 
block  angular  structure  arise  frequently  in  many  applications.  While 
much  work  has  been  devoted  to  exploiting  this  special  structure  when 
the  problem  variables  are  assumed  to  be  continuous,  little  considera- 
tion has  been  given  to  models  of  this  type  in  which  the  var_ables 
are  required  to  take  on  only  integer  values.  In  this  report,  an 
algorithm  for  the  decomposition  of  block  angular  integer  programs 
is  presented. 

The  block  angular  integer  program  consists  of  several  subproblems 
which  would  operate  independently  except  that  they  are  tied  together 
by  a set  of  linking  constraints.  Conceptually,  these  linking  con- 
straints are  viewed  as  representing  common  resources  which  the  sub- 
problems must  share.  The  problem  thus  becomes  that  of  determining 
an  optimal  allocation  of  these  resources  among  the  subproblems. 

Towards  this  end,  a branch-and-bound  search  routine  is  developed. 
It  is  shown  how  the  LP-optimal  dual  multipliers  and  any  slacks  which 
appear  in  the  optimal  integer  solutions  to  the  subproblems  can  be 
used  to  guide  the  search,  as  well  as  for  bounding  and  fathoming 
purposes.  Special  structures  which  arise  when  there  is  only  a single 
linking  constraint  are  discussed  in  detail. 

Since  the  problem  decomposes  completely  once  an  allocation  of  the 
linking  resources  is  specified,  only  the  subproblems  ever  need  be 
solved  explicitly.  Computational  results  obtained  with  the  decomposi- 
tion algorithm  are  reported. 
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